Non-intrusive monitoring and control of integrated circuits

ABSTRACT

A method of monitoring operations of a set of ICs. The method loads a first set of configuration data into a first IC for configuring a group of configurable circuits of the first IC to perform operations of a user design. The method receives a definition of an event based on values of a set of signals in the user design and a set of corresponding actions to take when the event occurs. The set of signals includes at least one signal received from a second IC. The method generates an incremental second set of configuration data based on the definition of the event and the set of corresponding actions. While the first IC is performing the operations of the user design, the method loads the incremental second set of configuration data into the first IC and monitors the signals received from the second IC at the first IC.

CLAIM OF BENEFIT TO PRIOR APPLICATIONS

The present application claims the benefit of and claims priority toU.S. patent application Ser. No. 14/824,368, entitled “Non-IntrusiveMonitoring and Control of Integrated Circuits.” filed Aug. 12, 2015;U.S. patent application Ser. No. 14/281,262, entitled, “Non-IntrusiveMonitoring and Control of Integrated Circuits,” filed May 19, 2014; U.S.Provisional Patent Application 61/842,966, entitled, “Non-IntrusiveMonitoring and Control of Integrated Circuits,” filed Jul. 4, 2013; U.S.Provisional Patent Application 61/843,921, entitled, “Non-IntrusiveMonitoring and Control of Integrated Circuits,” filed Jul. 9, 2013; andU.S. Provisional Patent Application 61/879,578, entitled, “Non-IntrusiveMonitoring and Control of Integrated Circuits.” filed Sep. 18, 2013. Thecontents of U.S. application Ser. Nos. 14/824,368 and 14/281,262 andU.S. Provisional applications 61/842,966, 61/843,921, and 61/879,578 arehereby incorporated by reference.

BACKGROUND

Programmable Logic Devices (“PLDs”) are configurable integrated circuits(“ICs”) which can be used to implement multiple circuit designs createdby users (“user designs”) without having to fabricate a new IC for eachdesign. However, due to the complexity of the systems being implemented,such user designs usually include various design bugs, design defects,or unexpected runtime behavior that pass unseen through design andtesting. Therefore, it is common for user designs to include debugfunctionality to aid designers and other users in identifying andcorrecting such bugs, defects, and behavior. Debug functionalitytypically includes software and hardware components that collectively orseparately are referred to as the debug network of the user design, withthe purpose of collecting run-time data to evaluate, detect and correctpossible bugs, defects or runtime behavior.

In some cases, the debug network is implemented by using theconfigurable circuits of the PLD. The primary circuit structure uses thesame circuits to implement the logic functionality specified within auser design. In such cases, the more complicated the debug network, thelarger the amount of PLD resources consumed, leaving fewer resources forimplementing the user design. As a result, user designs become lesssophisticated. Additionally, a change to the functionality of the debugnetwork will cause the entire IC design to have to be recompiled,downloaded, and loaded onto the IC. This is due to the fact that changesto a design, even when made on a small scale to localized circuits, willhave a design-wide impact affecting the overall circuit routing ortiming of the design. These changes also create the risk that thecircuit logic, including seemingly unrelated logic, may be “broken” dueto errors in implementing the new functional change. Because of thisrisk, extensive regression testing and verification of the logic of theprimary circuit structure and debug network is required.

In other cases, the debug network is fixed-function circuitry thatexists exclusively for debugging purposes. However, implementing thedebugging circuitry as fixed-function circuitry also has severaldrawbacks. For instance, resources are dedicated to performing debugfunctionality whether or not the user has a need for such debugfunctionality. A user design that has undergone extensive regressiontesting and verification before implementation may require only aminimal set of debug functionality. Similarly, a user design that isonly an incremental upgrade to an already existing and verified designwould have little use for the debug network. Therefore, the dedicatedresources of the debug network go unused and are effectively wasted asthese resources cannot be modified to complement the functionality ofthe primary circuit structure that implements the user design.

The fixed-function implementation of the debug network required systemdesigners to predict what functionality had to be included within thedebug network. System designers had to anticipate what statisticalmonitoring or debug functionality was needed in advance of designing thedebug network and deploying the PLD. Unanticipated usage, behavior, oroperating conditions in the field could pose issues beyond the debuggingscope of the programmed debug network, forcing users to have to employthird party tools or other means to perform the additional debugfunctionality needed to handle the unanticipated usage, behavior, oroperating conditions.

A further issue prevalent in traditional debug networks is the inabilityof the networks to provide meaningful debug data to the users. Debugnetworks often blindly report data at a debug point within the userdesign. In many instances, the reported data has to be manually parsedor stepped through to find relevant data points at which an erroroccurs. As a result, users waste time in deciphering the debug data.

BRIEF SUMMARY

Some embodiments provide a system and method for obtaining visibilityinto an in-the-field device. These embodiments provide a monitoring toolthat allows unobtrusive full read access to a running user design toenable rapid-response monitoring, debugging, and response to exceptionsand other events. In contrast to the existing monitoring solutions, thesystem of these embodiments does not require downtime (or stopping) ofthe device or recompilation of code to be run on the device, does notdisturb the functionality timing of a running design, and can be appliedas a hot-fix in the lab or in the field.

The device in some embodiments has a primary circuit structure thatincludes a group of reconfigurable circuits and a configuration andmonitoring network that operates in non-intrusive manner to theoperations of the primary circuit structure. Specifically, anon-intrusive configuration and monitoring network operation is onewhich does not need to use circuits that would otherwise be used toimplement the user's design. In some embodiments, the configuration andmonitoring network does not change any values of resources of theprimary circuit structure while the configuration and monitoring networkmonitors the primary circuit structure. Some advantages of anon-intrusive configuration and monitoring network of some embodimentsare that the non-intrusive configuration and monitoring network does notinterfere with the implementation of the user design in the primarycircuit structure and does not require restructuring the physicalimplementation of the user design in the primary circuit structure inorder to retrieve data from different parts of the circuit.

Some embodiments utilize the user signals that were expressed in theuser RTL in order to build conditions that fire the trigger signal.However, user signals as originally expressed in the RTL source code maynot be available on-chip due to synthesis optimization or rescaling.During synthesis optimization, user signals may have been eliminatedduring synthesis, or they may be contained within a component such as aLUT. Also, multiple fabric-cycle rate signals may need to be recombinedto create a single user-cycle rate user signal as a result of rescaling.

Some embodiments employ two different strategies to create signals thatcan be used for triggering: (1) user signals that were eliminated duringsynthesis are reconstructed by logically combining other existent fabricsignals with one or more LUTs and (2) multiple versions of a user signalthat are created by rescaling are recombined into a single fabric signalwith a rescaling mux.

Some embodiments provide a method of creating views for monitoring andcontrolling the operations of an IC. A view is a tool for specifyingevents and response to the events. Each view is a statement thatidentifies what to do (e.g., what data items to collect or what controlsto activate) when a condition becomes true. For instance, a user cancreate a view to start collecting data when a set of user design signalssatisfies a particular condition.

Views enable inspection of production designs in the field or in the labwhile the IC is performing operations of the user design. The use ofviews in some embodiments allows all application-visible states to beunobtrusively monitored and inspected. No recompilation or replacementof the code running on the IC and no pre-declaration of viewable signalsare required. Some embodiments utilize an interactive process fordescribing the views. The result of the process is a text file thatdescribes a view specification. Some embodiments use a proprietaryformat while other embodiments use a subset of System Verilog to defineview specifications.

Some embodiments provide an application-specific integrated circuit(ASIC) that includes a group of non-configurable circuits customized forperforming operations for a particular use. The ASIC also includes a setof reconfigurable circuits that configurably perform operations of auser design based on configuration data. The ASIC also includes aconfiguration and monitoring network to receive incremental sets ofconfiguration data while the set of reconfigurable circuits isperforming operations of the user design. Each incremental set of datais for configuring the configuration and monitoring network to monitor aset of signals received at the set of reconfigurable circuits from oneor more non-configurable circuits of the ASIC and to take a set ofactions when values of the monitored signals satisfy a predefinedcondition.

Some embodiments provide a method of monitoring operations of anintegrated IC that includes a set of configurable circuits forconfigurably performing a set of operations based on configuration data.The method loads a first set of configuration data into the IC toconfigure a group of configurable circuits to perform operations of auser design. The method receives a definition of an event based onvalues of a set of signals in the user design and a set of correspondingactions to take when the event occurs. The method generates anincremental second set of configuration data based on the definition ofthe event and the set of corresponding actions. The method, while the ICis performing the operations of the user design, loads the incrementalsecond set of configuration data into the IC to monitor for the eventand to take the set of actions when the event occurs.

Some embodiments provide a method of monitoring operations of a systemcomprising a set of ICs mounted on a circuit board. The set of ICsincludes a first IC that has a set of configurable circuits forconfigurably performing a set of operations based on configuration data.The method loads a first set of configuration data into the first IC forconfiguring a group of configurable circuits in the set of configurablecircuits to perform operations of a user design. The method receives adefinition of an event based on values of a set of signals in the userdesign and a definition of a set of corresponding actions to take whenthe event occurs. The set of signals includes at least one signal from asecond IC in the set of ICs received at an input port of the first IC.The method generates an incremental second set of configuration databased on the definition of the event and the set of correspondingactions. The method, while the first IC is performing the operations ofthe user design, loads the incremental second set of configuration datainto the first IC. The method, while the first IC is performing theoperations of the user design, monitors the signal received from thesecond IC at the input port of the first IC.

Some embodiments provide a method of monitoring operations of an IC thatincludes a set of configurable circuits for configurably performing aset of operations based on configuration data. The method loads a firstset of configuration data into the IC for configuring a group ofconfigurable circuits in the set of configurable circuits for performingoperations of a user design. The method receives a definition of a firstevent based on values of a first set of signals in the user design and afirst set of actions to take when the first event occurs. The methodgenerates an incremental second set of configuration data based on thedefinition of the first event and the first set of actions. The method,while the IC is performing the operations of the user design, loads theincremental second set of configuration data into the IC to monitor forthe first event and to take the first set of actions when the firstevent occurs. The method receives a definition of a second event basedon values of a second set of signals in the user design and a second setof actions to take when the second event occurs. The method generates anincremental third set of configuration data based on the definition ofthe second event and the second set of actions. The method, while the ICis performing the operations of the user design, loads the incrementalthird set of configuration data into the IC to monitor for the secondevent and to take the second set of actions when the second eventoccurs.

Some embodiment provide a non-transitory machine-readable medium thatstores a program for generating configuration data sets for monitoringoperations of a system comprising a set of ICs mounted on a circuitboard. The set of ICs includes a first IC that has a set of configurablecircuits for configurably performing a set of operations based onconfiguration data. The program is executable by at least one processingunit. The program includes a set of instructions for loading a first setof configuration data into the first IC to configure a group ofconfigurable circuits in the set of configurable circuits to performoperations of a user design. The program also includes a set ofinstructions for receiving a definition of an event based on values of aset of signals in the user design and a set of corresponding actions totake when the event occurs. The set of signals includes at least onesignal from a second IC in the set of ICs received to an input port ofthe first IC. The program also includes a set of instructions forgenerating an incremental second set of configuration data based on thedefinition of the event and the set of corresponding actions. Theprogram also includes a set of instructions for loading, while the firstIC is performing the operations of the user design, the incrementalsecond set of configuration data into the first IC. The program alsoincludes a set of instructions for monitoring, while the first IC isperforming the operations of the user design, the signal received fromthe second IC at the input port of the first IC.

The preceding Summary is intended to serve as a brief introduction tosome embodiments of the invention. It is not meant to be an introductionor overview of all inventive subject matter disclosed in this document.The Detailed Description that follows and the Drawings that are referredto in the Detailed Description will further describe the embodimentsdescribed in the Summary as well as other embodiments. Accordingly, tounderstand all the embodiments described by this document, a full reviewof the Summary, Detailed Description and the Drawings is needed.Moreover, the claimed subject matters are not to be limited by theillustrative details in the Summary, Detailed Description and theDrawing, but rather are to be defined by the appended claims, becausethe claimed subject matters can be embodied in other specific formswithout departing from the spirit of the subject matters.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purpose of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 conceptually illustrates the dynamic routing of user signals fromthe primary circuit structure to the configuration and monitoringnetwork in some embodiments.

FIG. 2 conceptually illustrates an example of a configurable IC thatincludes numerous configurable tiles.

FIG. 3 conceptually illustrates the configurable circuit architecture ofsome embodiments of the invention.

FIG. 4 illustrates an alternative tile structure that is used in someembodiments.

FIG. 5 conceptually illustrates an example of an IC with sub-cyclereconfigurable circuits (i.e., circuits that are reconfigurable on asub-cycle basis).

FIG. 6 provides an overview of a configuration and monitoring network ofsome embodiments interfacing with a primary circuit structure.

FIG. 7 conceptually illustrates a group of the units of theconfiguration and monitoring network of some embodiments of theinvention.

FIG. 8 conceptually illustrates a group of the units of theconfiguration and monitoring network some embodiments of the inventionin more detail.

FIG. 9 conceptually illustrates examples of different configurations forthe basic reconstruction units.

FIG. 10 conceptually illustrates the reconstruction circuitry in someembodiments of the invention where the number of component signalsneeded to create the user signal is up to 3 and the rescaling factor is4.

FIG. 11 conceptually illustrates chaining of multiple deskew lines insome embodiments of the invention.

FIG. 12 conceptually illustrates a process for utilizing an overlaycreated from a view to perform an action when an event occurs in someembodiments of the invention.

FIG. 13 conceptually illustrates an example of in-the-lab operation modefor an on-board configured IC in some embodiments of the invention.

FIG. 14 conceptually illustrates an example of in-the-lab operation modefor an embedded configuration in some embodiments of the invention.

FIG. 15 conceptually illustrates a system for generating and using viewsin some embodiments of the invention.

FIGS. 16A and 16B illustrate examples of user interfaces for generatingviews in some embodiments of the invention.

FIG. 17 conceptually illustrates a waveform viewer launched to displaywaveform representation of signals being traced after an event specifiedin a view is triggered in some embodiments of the invention.

FIG. 18 conceptually illustrates a process for using views in a labenvironment in some embodiments of the invention.

FIG. 19 conceptually illustrates a localized in-the-field configurationof an IC in some embodiments of the invention.

FIG. 20A conceptually illustrates an extended in-the-field configurationof an IC in some embodiments of the invention.

FIG. 20B conceptually illustrates an extended in-the-field configurationof an IC in some embodiments of the invention.

FIG. 21 conceptually illustrates a localized in-the-field configurationof an embedded configuration of some embodiments of the invention.

FIG. 22 conceptually illustrates an extended in-the-field configurationof an embedded configuration of some embodiments of the invention.

FIG. 23 conceptually illustrates a process for operating an IC in thefield in some embodiments of the invention.

FIG. 24 conceptually illustrates a process for triggering events andtaking actions defined by a view in some embodiments of the invention.

FIG. 25 conceptually illustrates a process for performing root causeanalysis in some embodiments of the invention.

FIG. 26 conceptually illustrates a process for performing root causeanalysis in some embodiments of the invention.

FIG. 27 conceptually illustrates a process for performing in-the-fieldfault injection in some embodiments of the invention.

FIG. 28 conceptually illustrates a process for packet monitoring in someembodiments of the invention.

FIG. 29 conceptually illustrates a data center in some embodiments ofthe invention.

FIG. 30 conceptually illustrates tracking of packets between source anddestination nodes in a network according to some embodiments of theinvention.

FIG. 31 conceptually illustrates the network of FIG. 30 where a packethas not arrived at an intended destination.

FIG. 32 conceptually illustrates the delivery of enhanced monitoring anddiagnostic tools from device manufacturers to the end users in someembodiments of the invention.

FIG. 33 conceptually illustrates a process to verify a designspecification during simulation.

FIG. 34 conceptually illustrates a process for using the same assertionsthat were used during simulation in the field without recompiling orreplacing the code loaded in an IC in some embodiments of the invention.

FIG. 35 conceptually illustrates an ASIC with a configurable device insome embodiments of the invention.

FIG. 36 conceptually illustrates an electronic system with which someembodiments of the invention are implemented.

DETAILED DESCRIPTION

In the following detailed description of the invention, numerousdetails, examples, and embodiments of the invention are set forth anddescribed. However, it will be clear and apparent to one skilled in theart that the invention is not limited to the embodiments set forth andthat the invention may be practiced without some of the specific detailsand examples discussed.

I. Terms and Definitions

A. Integrated Circuits (ICs)

Some embodiments of the invention monitor and control the operations ofan IC. An IC is a device that includes numerous electronic components(e.g., transistors, resistors, capacitors, diodes, etc.) that aretypically embedded on the same substrate, such as a single piece ofsemiconductor wafer. These components are connected with one or morelayers of wiring to form multiple circuits, such as Boolean gates,memory cells, arithmetic units, controllers, decoders, etc. An IC isoften packaged as one chip in a single IC package, although some ICpackages include multiple pieces of substrate or wafer.

A design layout is a geometric description of the circuit componentsincluded in an IC's design. An IC's design layout is often obtained byusing a set of computer-based electronic design automation tools (EDAs)to transform a code representation (e.g., a register transfer level(RTL) representation) or circuit representation of the design into ageometric description. The design process entails various operations.Some conceptual representations for some of the various physical-designoperations that EDA applications perform to obtain the IC layoutsinclude: (1) circuit partitioning, which partitions a circuit if thecircuit is too large for a single chip; (2) floor planning, which findsthe alignment and relative orientation of the circuit modules; (3)synthesis, which transforms an RTL or circuit representation to anothercircuit representation that is mapped to a particular technology of aparticular IC; (4) layout, which generates the physical design (orlayout) of the IC which includes placement and routing for defining thepositions of the circuit modules and the interconnects between thecircuit modules; (5) power optimization, which is done to reduce thepower consumption of the design; and (6) verification, which checks thelayout to ensure that it meets design and functional requirements. Itshould be apparent to one of ordinary skill in the art that in someembodiments the order in which the various EDA operations are performedneed not adhere to the presentation order of the conceptualrepresentations above.

B. Configurable IC Architecture

Some embodiments of the invention monitor and control the operations ofan IC with configurable circuits. A configurable circuit is a circuitthat can “configurably” perform a set of operations. Specifically, aconfigurable circuit receives a configuration data set that specifiesthe operation that the configurable circuit must perform from the set ofoperations that it can perform. In some embodiments, configuration datais generated outside of the IC. In these embodiments, a set of softwaretools typically converts a high-level IC design (e.g., a circuitrepresentation or a hardware description language design) into a set ofconfiguration data bits that can configure the configurable circuits ofthe IC to implement the IC design.

Examples of configurable circuits include configurable logic circuitsand configurable interconnect circuits. A logic circuit is a circuitthat can perform a function on a set of input data that it receives. Aconfigurable logic circuit is a logic circuit that can be configured toperform different functions on its input data set.

A configurable interconnect circuit is a circuit that can configurablyconnect an input set to an output set in a variety of ways. Aninterconnect circuit can connect two terminals or pass a signal from oneterminal to another by establishing an electrical path between theterminals. Alternatively, an interconnect circuit can establish aconnection or pass a signal between two terminals by having the value ofa signal that appears at one terminal appear at the other terminal. Inconnecting two terminals or passing a signal between two terminals, aninterconnect circuit in some embodiments might invert the signal (i.e.,might have the signal appearing at one terminal inverted by the time itappears at the other terminal). In other words, the interconnect circuitof some embodiments implements a logic inversion operation inconjunction to its connection operation. Other embodiments, however, donot build such an inversion operation in any or all of theirinterconnect circuits.

The configurable interconnect circuit passes signals through a routingfabric of the configurable IC. The routing fabric provides acommunication pathway for routing signals to and from source anddestination circuits or components. In some embodiments, the routingfabric includes storage elements in addition to the various routingcircuits, the wire segments (e.g., the metal or polysilicon segments)that connect to the routing circuits, and vias that connect to thesewire segments and to the terminals of the routing circuits. Thesestorage elements include latches and registers distributed across therouting fabric that provide one or more different means for storingsignals in the routing fabric.

In some of these embodiments, the routing fabric also includes buffersfor achieving one or more objectives (e.g., maintaining the signalstrength, reducing noise, altering signal delay, etc.) vis-a-vis thesignals passing along the wire segments. In conjunction with, or insteadof, these buffer circuits, the routing fabric of some embodiments mightalso include one or more non-configurable circuits (e.g.,non-configurable interconnect circuits).

The IC of some embodiments includes configurable logic circuit andconfigurable interconnect circuits for routing the signals to and fromthe configurable logic circuits. An IC with configurable circuits issometimes referred to a configurable IC. However, in addition toconfigurable circuits, the IC also typically includes non-configurablecircuit (e.g., non-configurable logic circuits, interconnect circuits,memories, etc.).

In some embodiments, the configurable resources (e.g., configurablelogic resources, routing resources, memory resources, etc.) that aregrouped in conceptual configurable tiles are arranged in several rowsand columns. Together, this arrangement forms a primary circuitstructure of the IC that implements the user design logic.

In addition to this primary circuit structure of the IC, someembodiments further provide a secondary IC network (also referred toherein as configuration and monitoring network) that is “on-chip.” Insome embodiments, the on-chip configuration and monitoring network is anetwork of resources that is located on the same physical wafer as theresources of the primary circuit structure. In some embodiments, theon-chip configuration and monitoring network is a network of resourcesthat is located on a different physical wafer or layer than the primarycircuit structure, but the wafers or layers for both the primary circuitstructure and configuration and monitoring network are included withinthe same physical package enclosing the IC as a single chip.Accordingly, the below described functionality of the configuration andmonitoring network is implemented and performed on the same physicalchip as the primary circuit structure. In some embodiments, theconfiguration and monitoring network is an optical network, while theprimary circuit structure is an electrical network.

In some embodiments, the configuration and monitoring network is adifferent network than the primary circuit structure implementing theuser design. Specifically, in some embodiments, the user design is notmapped to the configuration and monitoring network. Rather, theconfiguration and monitoring network of some embodiments is aconfiguration network, debug, and monitoring network that providesfunctionality extended beyond traditional debug functionality.

When providing configuration functionality, the configuration andmonitoring network is the means through which configuration data that isstreamed into the IC is routed to the appropriate tiles and ultimatelyto the appropriate configurable circuits of the primary circuitstructure that configure to perform operations in accordance with theuser design. When providing debug functionality, the configuration andmonitoring network can be used to diagnose and isolate issues within theprimary circuit structure. Such functionality may operate independentof, and/or complement the functionality of, the user design implementedby the primary circuit structure. In each instance, the configurationand monitoring network operates in a non-interfering manner with theoperations of the primary circuit structure.

The configuration and monitoring network interfaces with the primarycircuit structure through a set of bitlines that pass through and areshared amongst various tiles of configurable circuits of the primarycircuit structure. In some embodiments, relevant user signals aredynamically routed over the bitlines from the primary circuit structureto the configuration and monitoring network and from the configurationand monitoring network to the primary circuit structure such that thereis no impact to the user circuits (e.g., the configurable circuitsimplementing the user design) configured in the primary circuitstructure. Accordingly, there is no impact to the functionalityconfigured within the primary circuit structure (i.e., the user design).

In some embodiments, the configuration and monitoring network isinitially configured via an external interface into the IC. In someembodiments, the external interface includes Joint Test Action Group(“JTAG”) interface, flash, slave peripheral port, or through other meansof communications with the IC, such as the I/O buffers of the IC. Also,in some embodiments, these various external interfaces may be used toperform read-back from the configuration and monitoring network to theexternal interfaces. In addition to providing access to theconfiguration and monitoring network from outside of the IC, someembodiments of the IC include a “fabric port,” through which a usercircuit, or user logic, of the primary circuit structure accesses theconfiguration and monitoring network. In some embodiments, the usercircuit includes logic that is not implemented on either the primarycircuit structure or the configuration and monitoring network, but mayinclude logic in the same package or IC of a System-On-Chip (“SoC”).

FIG. 1 conceptually illustrates the dynamic routing of user signals fromthe primary circuit structure to the configuration and monitoringnetwork in some embodiments. As shown, an integrated circuit (“IC”) 105includes the primary circuit structure 110 and the configuration andmonitoring network 115 with various interconnects 170-180 that allow forintercommunications between the two networks.

The primary circuit structure 110 includes blocks of configurablecircuits 120-145 that represents tiles of the IC. The variousinterconnects within the primary circuit structure 110 connect the block120-145 to one another. Additionally, these interconnects also includebitlines for passing signals to the configuration and monitoringnetwork. For instance, a communication pathway between the primarycircuit structure 110 and configuration and monitoring network 115exists at locations 170-180. These locations may include unused storageelements within the routing fabric or routing circuits from whichsignals passing through the primary circuit structure 10 reach thecircuits of the configuration and monitoring network 115.

As such, signals may pass from the primary circuit structure to theconfiguration and monitoring network in a manner that does not interferewith the operation of the primary circuit structure. As shown, theconfiguration and monitoring network 115 includes circuits 160-165 witha separate set of interconnects over which signals from thecommunication bitlines with the primary circuit structure pass into thecircuits 160-165 of the configuration and monitoring network.

In order to illustrate the conceptual difference between the primarystructure and the configuration and monitoring network, the primarycircuit structure 110 and the configuration and monitoring network 115are shown as being separate in this figure. However, in someembodiments, the circuits and bitlines of the configuration andmonitoring network are physically interspersed with the circuits andbitlines of the primary circuit structure. In other words, theconfiguration and monitoring network may be thought of as an “overlay”network with regard to the primary circuit structure.

FIG. 2 conceptually illustrates an example of a configurable IC 200 thatincludes numerous configurable tiles 205. The configurable tiles 205communicate with each other through the routing fabric of the IC. Asmentioned above, these configurable tiles 205 form a primary circuitstructure of the IC. Each configurable tile 205 receives a set of lines210 that are part of the configuration and monitoring network. The lines210 pass debug data, configuration data, or other data (e.g., resourcestate data, assertions, logic computations, etc.) on to transportnetwork 215 of the configuration and monitoring network, which in turnpasses the data on to other components of the configuration andmonitoring network (not shown). In some embodiments, the lines 210 alsopass data from the configuration and monitoring network to the primarycircuit structure.

In some embodiments, the set of lines 210 are a uniform set of linesdistributed throughout the primary circuit structure, through every setof tiles. The set of lines 210 may include 18 lines, six of which areused to provide control signals and twelve of which are used to providedata signals. The six control signals serve as an opcode (operationcode), while the twelve signals serve as the operand (i.e., dataargument) associated with the opcode. Some examples of opcodes andoperands are further discussed below. While this specification discussesspecific examples with respect to the width of bitlines and data packets(e.g., 18-bit bitlines, 18-bit data frames, six-bit opcodes, twelve-bitoperands, etc.), a person of ordinary skill in the art would recognizethat these are merely illustrative examples, and that any other numberof bits can be used without departing from the spirit of the invention.

In some embodiments, there is an unused area of the IC between theconfigurable tiles 205 and the transport network 215. Having thetransport network 215 be separate from the main set of configurablecircuits allows multiple generations of the configurable IC to usedifferent designs for the transport network 215 without disrupting thedesign of the fabric of the primary circuit structure. Some embodimentsuse a packet switching technology to route data to and from theresources in the configurable tiles. Hence, over the lines 210, theseembodiments can route variable length data packets to each configurabletile in a sequential or random access manner. Additionally, the packetswitching allows the lines 210 to be shared by all tiles and circuits ofthe primary circuit structure in communications with the configurationand monitoring network.

Data packets muted according to the packet switching functionality ofsome embodiments include one or more data frames. In some embodiments,an initial set of frames (e.g., first one or two frames) of the packetidentifies configurable tiles for routing the remaining frames of thedata packet. In other words, the initial set of frames specifies one ormore destinations for receiving the data packet. Some embodiments allowtiles to be individually addressed, globally addressed, or addressedbased on their tile types. The remaining frames can then containconfiguration, debug, or other data for performing one or more overlayapplications of the configuration and monitoring network.

In some embodiments, the configurable circuits might be organized in anarrangement that has all the circuits organized in an array with severalaligned rows and columns. In addition, within such a circuit array, someembodiments disperse other circuits (e.g., memory blocks, processors,macro blocks, IP blocks, controllers, clock management units, etc.).FIGS. 3-4 illustrate several configurable circuitarrangements/architecture in some embodiments of the invention. One sucharchitecture is illustrated in FIG. 3.

FIG. 3 conceptually illustrates the configurable circuit architecture ofsome embodiments of the invention. As shown in FIG. 3, this architectureis formed by numerous configurable conceptual tiles that are arranged inan array with multiple rows and columns. It should be noted that in someembodiments a “conceptual tile” (or “tile” for short) does not denoteany physically distinct object, but is rather a way of referring togroups of circuitry in a repeated or nearly repeated pattern. In suchembodiments, the lines around individual tiles represent conceptualboundaries, not physical ones.

In FIG. 3, each configurable tile is a configurable logic tile, which,in this example, includes one configurable three-input logic circuit310, three configurable input-select interconnect circuits 315, andeight configurable routing interconnect circuits 320. For eachconfigurable circuit, the configurable IC 300 includes a set of storageelements for storing a set of configuration data. In some embodiments,the logic circuits are look-up tables (LUTs) while the interconnectcircuits are multiplexers. In this specification, many embodiments aredescribed as using multiplexers. It will be clear to one of ordinaryskill in the art that other embodiments can be implemented with inputselection circuits other than multiplexers. Therefore, any useof“multiplexer” in this specification should be taken to also disclosethe use of any other type of input selection circuits.

In FIG. 3, an input-select multiplexer (“IMUX”) 315 is an interconnectcircuit associated with the LUT 310 that is in the same tile as theinput select multiplexer. One such input select multiplexer (1) receivesseveral input signals for its associated LUT, and (2) based on itsconfiguration, passes one of these input signals to its associated LUT.

In FIG. 3, a routing multiplexer (“RMUX”) 320 is an interconnect circuitthat connects other logic and/or interconnect circuits. The interconnectcircuits of some embodiments route signals between logic circuits, toand from I/O circuits, and between other interconnect circuits. Unlikean input select multiplexer of some embodiments (which provides itsoutput to only a single logic circuit, i.e., which has a fan-out of only1), a routing multiplexer of some embodiments is a multiplexer that (1)can provide its output to several logic and/or interconnect circuits(i.e., has a fan-out greater than 1), or (2) can provide its output toother interconnect circuits. The RMUX receives several inputs and basedon its configuration, selects the input to pass along the output.

In the architecture illustrated in FIG. 3, each configurable logic tileincludes one three-input LUT, three input-select multiplexers, and eightrouting multiplexers. Other embodiments, however, might have a differentnumber of LUTs in each tile, different number of inputs for each LUT,different number of input-select multiplexers, and/or different numberof routing multiplexers. Other embodiments might also use differenttypes of logic circuits and/or interconnect circuits. Several sucharchitectures are further described in the U.S. Pat. No. 7,295,037,issued on Nov. 13, 2007.

Some of the configurable logic tiles of FIG. 3 together conceptuallyform configurable memory tiles, which are (1) tiles with blocks ofmemory, or (2) tiles that are adjacent to blocks of memory. FIG. 3illustrates two examples of configurable memory tiles. The first exampleis a memory tile 335 that is formed by a set of four aligned tiles thathave a memory block 330 in place of their four LUTs. In the secondexample, a memory tile 345 is formed by 16 tiles that neighbor a memoryblock 340. In the configurable logic tiles of the memory tiles 335 and345, the input select and routing interconnects serve as configurableports of the memory blocks.

In some embodiments, the examples illustrated in FIG. 3 represent theactual physical architecture of a configurable IC. However, in otherembodiments, the examples presented in FIG. 3 topologically illustratethe architecture of a configurable IC (i.e., they show arrangement oftiles, without specifying a particular physical position of thecircuits). In some embodiments, the position and orientation of thecircuits in the actual physical architecture of a configurable IC isdifferent from the position and orientation of the circuits in thetopological architecture of the configurable IC. Accordingly, in theseembodiments, the IC's physical architecture appears quite different fromits topological architecture.

In some embodiments, the configuration and monitoring network shares oneor more resources with the primary circuit structure to facilitate oneor more of the interfaces with the primary circuit structure. Theseresources include user design state (“UDS”) elements. UDS elements areelements that store values. At any particular time, the values stored bythe UDS elements define the overall user-design state of the primarycircuit structure at that particular time. In some embodiments, a UDSelement is capable of continuously outputting the value it stores.Examples of such elements include traditional latches, registers, userflip-flops, and memory structures. U.S. Pat. No. 7,224,181, issued onMay 29, 2007; U.S. Pat. No. 7,521,959, Issued on Apr. 21, 2009, and U.S.Pat. No. 8,456,190, issued on Jun. 4, 2013, describe other user-designstate elements that include routing multiplexers (“RMUXs”) that canserve as storage elements, RMUXs that have storage elements in feedbackpaths between their outputs and inputs, and storage elements at otherlocations in the routing fabric (e.g., between RMUXs).

More specifically, some embodiments have RMUXs where at least some ofthe RMUXs have state elements integrated at the output stage of the RMUXitself. Such RMUXs are referred to as routing circuit latches or RCLs.For instance, some RMUXs use complementary passgate logic (“CPL”) toimplement a routing multiplexer. Some of these embodiments thenimplement a routing multiplexer that can act as a latch by placingcross-coupled transistors at the output stage of the routingmultiplexer. Such an approach is further described in U.S. Pat. No.7,342,415, issued on Mar. 11, 2008. In the discussion below, routingmultiplexers that can serve as latches are referred to asrouting-circuit latches (“RCLs”).

In conjunction or instead of such RCLs, other embodiments utilize otherstorage elements for storing UDS data at other locations in theconfigurable routing fabric of a configurable IC. For instance, inaddition to or instead of having a storage element in the input and/oroutput stage of an RMUX, some embodiments place a storage element (e.g.,latch or register) in a feedback path between the output and input ofthe RMUX.

Some such UDS elements operate as transparent latches referred to as“time vias” (“TVs”) or clock driven latches referred to as “conduits.”When a TV is “open,” the TV's output value immediately assumes the TV'scurrent input value. In other words, the TV acts as a wire (with someadditional delay). When the TV closes, it captures and holds the currentoutput value (i.e., the output no longer follows the input).

Some or all of these TVs can be accessed via the configuration andmonitoring network in one of two modes: active mode and passive (ortrace) mode. Active mode allows users to read and write stored values inany circuit of the IC, including closed TVs (open TVs do not storevalues) while the circuit is stopped. Passive mode continuouslytransmits TV values to the configuration and monitoring network in realtime. These modes are further described below. In some embodiments, thistransmission of TV values occurs at the maximum user clock rate. Oncereceived by the configuration and monitoring network, these signalvalues can be stored in a trace buffer for later display and analysis.

Conduits, unlike TVs, introduce delay when performing a storageoperation. In some embodiments, conduits are implemented as singleedge-triggered flip-flops. In some embodiments, multiple conduits arechained together to provide longer delays, as necessary. In someembodiments, conduits are accessed in the same manner as TVs. In someembodiments, conduits are readable, writeable, and/or stream-able fromthe configuration and monitoring network.

In some embodiments, some or all of the latches, registers, TVs, orconduits are separate from the RMUXs of the routing fabric and areinstead at other locations in the routing fabric (e.g., between the wiresegments connecting to the outputs and/or inputs of the RMUXs). Forinstance, in some embodiments, the routing fabric includes a paralleldistributed path for an output of a source routing circuit to adestination circuit. A first path of the parallel distributed path,directly routes the output of the source routing circuit to a firstinput of the destination circuit. A second path running in parallel withthe first path passes the output of the source routing circuit through aUDS element before reaching a second input of the destination circuit.The storage element stores the output value of the routing circuit whenenabled. In some embodiments, the second path connects to a differentdestination component than the first path. When the routing fabricincludes buffers, some of these embodiments utilize these buffers aswell to build such latches, registers, TVs, or conduits.

In some embodiments, the configuration and monitoring network connectsto some or all of the UDS elements (e.g., latches, registers, memories,etc.) of the primary circuit structure to establish the communicationpathway between the two networks. In some embodiments, the configurationand monitoring network has a streaming mode that can direct variouscircuits in one or more configurable tiles of the primary circuitstructure to stream out their data during the operation of theconfigurable IC. In some embodiments, the determination of whichcircuits are to stream out their data is made before runtime of the IC.As discussed below, in some such embodiments, configuration data isloaded into the IC that identifies these circuits that are identifiedfor streaming. Accordingly, in some embodiments where the configurationand monitoring network connects to some or all of the UDS elements, theconfiguration and monitoring network can be used in a streaming mode tostream out data from the UDS elements of the tiles, in order to identifyany errors in the operation of the IC. In other words, the streaming ofthe data from the UDS elements can be used to debug the operation of theconfigurable IC.

In various places in this specification, signals or data are describedas going to the configuration and monitoring network from logiccircuits. RMUXs, and/or IMUXs of the primary circuit structure. In someembodiments, such data goes directly from the indicated circuits of theprimary circuit structure to the configuration and monitoring networkwithout any further intervening circuits. In other embodiments, data canbe sent from logic circuits, RMUXs or IMUXs of the primary circuitstructure through some type of intervening circuit (e.g., a stateelement). It will be clear to one of ordinary skill in the art thatreferences to data going to the configuration and monitoring networkfrom a circuit encompass both data going directly to a configuration andmonitoring network and data going to a configuration and monitoringnetwork through intervening circuits.

In some embodiments, the signals from circuits or tiles of the primarycircuit structure are conveyed in real time to various circuit elementsor circuit blocks of the configuration and monitoring network such thatthe configuration and monitoring network is able to always observe theprimary circuit structure during operation of the primary circuitstructure. For instance, a configuration and monitoring network thatcollects statistics regarding the performance of the primary circuitstructure will receive the signals at one or more counters of theconfiguration and monitoring network that measure the activity of therouted signals in the primary circuit structure.

Some embodiments might organize the configurable circuits in anarrangement that does not have all the circuits organized in an arraywith several aligned rows and columns. Therefore, some arrangements mayhave configurable circuits arranged in one or more arrays, while otherarrangements may not have the configurable circuits arranged in anarray.

Some embodiments might utilize alternative tile structures. Forinstance, FIG. 4 illustrates an alternative tile structure that is usedin some embodiments. This tile 400 has four sets 405 of 4-aligned LUTsalong with their associated IMUXs. It also includes eight sets 410 ofRMUXs and eight banks 415 of configuration RAM storage. Each 4-alignedLUT tile shares one carry chain. One example of which is described inU.S. Pat. No. 7,295,037, entitled “Configurable IC with Routing Circuitswith Offset Connections”, issued on Nov. 13, 2007. One of ordinary skillin the art would appreciate that other organizations of LUT tiles mayalso be used in conjunction with the invention and that theseorganizations might have fewer or additional tiles.

C. Reconfigurable IC Architecture

Some embodiments of the invention perform placement for an IC that hasreconfigurable circuits that reconfigure (i.e., base their operation ondifferent sets of configuration data) one or more times during theoperation of the IC. Specifically, these ICs are configurable ICs thatcan reconfigure one or more circuits during runtime. These IC typicallyincludes reconfigurable logic circuits and/or reconfigurableinterconnect circuits, where the reconfigurable logic and/orinterconnect circuits are configurable logic and/or interconnectcircuits that can “reconfigure” more than once at runtime. Aconfigurable logic or interconnect circuit reconfigures when it basesits operation on a different set of configuration data. An IC withreconfigurable circuits is sometimes referred to as a reconfigurable IC.However, in addition to reconfigurable circuits, the IC also typicallyincludes non-configurable circuits (e.g., non-configurable logiccircuits, interconnect circuits, memories, configurable circuits thatare not sub-cycle reconfigurable, etc.).

In some embodiments, the logic circuits are look-up tables while theinterconnect circuits are multiplexers. Also, in some embodiments, theLUTs and the multiplexers are sub-cycle reconfigurable circuits(sub-cycles of reconfigurable circuits may be alternatively referred toas “reconfiguration cycles”). In some of these embodiments, the IC withconfigurable circuits stores multiple sets of configuration data for asub-cycle reconfigurable circuit, so that the reconfigurable circuit canuse a different set of configuration data in different sub-cycles. Areconfigurable circuit of some embodiments that operates on four sets ofconfiguration data receives its four configuration data setssequentially in an order that loops from the first configuration dataset to the last configuration data set. Such a sequentialreconfiguration scheme is referred to as a 4 “loopered” scheme. Otherembodiments, however, might be implemented as six or eight looperedsub-cycle reconfigurable circuits. In a six or eight looperedreconfigurable circuit, a reconfigurable circuit receives six or eightconfiguration data sets in an order that loops from the lastconfiguration data set to the first configuration data set. Sub-cyclereconfigurable circuits are also referred to as spacetime reconfigurablewhile reconfigurable circuits that are not sub-cycle reconfigurable arereferred to as spatial reconfigurable circuits.

FIG. 5 conceptually illustrates an example of an IC with sub-cyclereconfigurable circuits (i.e., circuits that are reconfigurable on asub-cycle basis). In this example, the IC implements an IC design 505that operates at a clock speed of X MHz. The operations performed by thecomponents in the IC design 505 can be partitioned into four sets ofoperations 520-535, with each set of operations being performed at aclock speed of X MHz.

FIG. 5 then illustrates that these four sets of operations 520-535 canbe performed by one IC 510 with sub-cycle reconfigurable circuits. TheIC operates at 4X MHz. In some embodiments, four cycles of the 4X MHzclock correspond to four sub-cycles within a cycle of the X MHz clock.Accordingly, this figure illustrates the IC 510 (i.e., at least one orthe reconfigurable circuits of the IC) reconfiguring four times duringfour cycles of the 4X MHz clock (i.e., during four sub-cycles of the XMHz clock). During each of these reconfigurations (i.e., during eachsub-cycle), the IC 510 performs one of the identified four sets ofoperations 520-535. In other words, the faster operational speed of theIC 510 allows the circuits of this IC to reconfigure four times duringeach cycle of the X MHz clock, in order to perform the four sets ofoperations 520-535 sequentially at a 4X MHz rate instead of performingthe four sets of operations in parallel at an X MHzrate.

Several embodiments were described above by reference to examples ofsub-cycle reconfigurable circuits that operate based on four differentsets of configuration data. In some of these examples, a reconfigurablecircuit receives its four different configuration data sets sequentiallyin an order that loops from the last configuration data set to the firstconfiguration data set. Such a sequential reconfiguration scheme isreferred to as a 4-loopered scheme. Higher order loopered schemes (e.g.,8, 16, 32, etc.,) are likewise implemented in some embodiments.

While the reconfigurable circuits described above reconfigure insub-cycles of a user design clock cycle, one of ordinary skill in theart will understand that in some embodiments, the reconfiguration cyclesare not part of a larger user design clock cycle. Accordingly, anyfeatures described herein as using sub-cycles can also be implemented insome embodiments with reconfiguration cycles that are not sub-cycles ofa longer user design clock cycle. In some such embodiments, multiplereconfigurations of the reconfigurable circuits are performed cyclicallybased on a reconfiguration clock cycle. In some such embodiments, somereconfigurable circuits reconfigure sequentially through a sequence ofconfigurations over the course of multiple reconfiguration cycles, andthen repeat the sequence of configurations multiple times.

D. Rescaling

For a design of an integrated circuit (IC) that includes an original setof circuits designed to operate at a frequency F₀, rescaling is anoperation that transforms the original set of circuits into a rescaledset of circuits that can operate at a fractional frequency F₀/k andstill retain the functionality, latency, bandwidth and interface of theoriginal set of circuits.

Rescaling transforms the original set of circuits into the rescaled setof circuits by making multiple copies of the original set of circuits. Arescaling operation with a rescaling factor of k makes k replicas orcopies of the original set of circuits. Each replica set of circuits (orreplicas) includes counterpart components and connections that areidentical to components and connections in the original set of circuits.Thus for an original set of circuits that includes sequential elements(e.g., flip-flops) and combinational elements (e.g., logic gates andmultiplexers), rescaling creates replicas that include counterpartsequential and combinational elements of the sequential andcombinational elements of the original.

In some embodiments, the rescaling operation also establishesconnections between the replica sets of circuits according to logicalequivalency and phase relationships. For an original set of circuitsthat includes sequential elements operating at certain clock phases,rescaling transforms those clock phases into fractional phases at thecounterpart sequential elements in the replicas. The fractional phase φ′at each sequential element in a replica is thus calculated according tothe equationφ′=(φ+360°×i)/k  (1)where i is the index of a particular replica (the replicas are indexedas 0 . . . k−1), and φ′ is the fractional phase for a counterpartsequential element in the particular replica set within the rescaled setof circuits.

Based on the calculated fractional phase φ′, some embodiments rewireconnections in the rescaled set of circuits such that each sequentialelement in each replica receives a functional equivalent input at aphaseφ′_(input)=φ′−Δ(φ,φ_(input))/k  (2)where φ_(input) is the phase of the input to the sequential elementbefore rescaling, and Δ (φ,φ_(input)) is the phase difference betweenthe sequential element and its input before rescaling. For a sequentialelement whose phase difference with its input in the original set ofcircuits is one entire cycle (i.e., Δ (Ψ, Ψ_(input))=360°), equation (2)becomes:φ′_(input)=(φ+360°×(i−1))/k  (3)

Since a functional equivalent input at a phase that satisfies equation(3) is often found in another replica set of circuits in the rescaledset of circuits, some embodiments of the rescaling operation rewire someor all sequential elements in a replica to receive inputs from otherreplicas. Some embodiments also add additional sequential elements intothe rescaled netlist in order to provide logically equivalent inputs tosequential elements at correct phase relationships.

The rewiring of the k replicas results in a rescaled set of circuitsthat has k parallel paths that each has a latency of 1/k cycles of theoriginal set of circuits. In some embodiments, each path runs at afractional frequency of F₀/k so the effective latency of the k paths isequivalent to the original set. Since there are k parallel paths runningat the fractional frequency of F₀/k, the effective bandwidth ismaintained.

To complete rescaling, some embodiments replace the original set ofcircuits with the rescaled set of circuits. Replacing the original setwith the rescaled set involves disconnecting the original set fromcertain peripheral nodes and connecting those peripheral nodes to therescaled set. Rescaling is further described in U.S. Patent Publication2012/0176155, entitled “Rescaling,” filed on Mar. 21, 2012.

II. Configuration and Monitoring Network Circuits

Some embodiments provide a system and method for obtaining visibilityinto an in-the-field device. These embodiments provide a monitoring toolthat allows unobtrusive full read access to a running user design toenable rapid-response monitoring, debugging, and response to exceptionsand other events. In contrast to the existing monitoring solutions, thesystem of these embodiments does not require downtime (or stopping) ofthe device or recompilation of code to be run on the device, does notdisturb the functionality timing of a running design, and can be appliedas a hot-fix in the lab or in the field.

The device in some embodiments has a primary circuit structure thatincludes a group of reconfigurable circuits and a configuration andmonitoring network that operates in non-intrusive manner to theoperations of the primary circuit structure. Specifically, anon-intrusive configuration and monitoring network operation is onewhich does not need to use circuits that would otherwise be used toimplement the user's design. In some embodiments, the configuration andmonitoring network does not change any values of resources of theprimary circuit structure while the configuration and monitoring networkmonitors the primary circuit structure. Some advantages of anon-intrusive configuration and monitoring network of some embodimentsare that the non-intrusive configuration and monitoring network: 1) doesnot interfere with the implementation of the user design in the primarycircuit structure and 2) does not require restructuring the physicalimplementation of the user design in the primary circuit structure inorder to retrieve data from different parts of the circuit.

In some embodiments, non-intrusive configuration and monitoring networkdoes not use circuits that are assigned to implement the user design inthe primary circuit structure, but the non-intrusive configuration andmonitoring network of some embodiments is uses “dedicated” circuits, forexample, configurable interconnect circuits. Therefore, once a userdesign circuit has been implemented on the primary circuit structure,such configurable circuit elements of the primary circuit structure thatare not used to implement the user design circuit may be put to use tosupport the configuration and monitoring network and transport network.

FIG. 6 provides an overview of a configuration and monitoring network ofsome embodiments interfacing with a primary circuit structure. As shownin this figure, this configuration and monitoring network includes a bus605 and a controller 615. FIG. 6 also shows a tile array 610 of theprimary circuit structure that includes multiple tiles. Each tileincludes one or more sets of decoders 690 and a pipeline register 665.This figure also shows a set of merge tiles 650 of a transport network692, bitlines 652, a trace buffer 660, deskew and reconstructioncircuitry 670, and trigger circuitry 680.

The bus 605 passes through each tile of the tile array 610 of theprimary circuit structure, so that the controller 615 can route packetsto the tiles of the tile array 610. In some embodiments, the controller615 is a microprocessor or some other circuit (e.g., a set ofconfigurable circuits of the IC configured as a controller that iscapable of performing the operations described below). In someembodiments, the controller 615 includes an interface (e.g., JTAG, orsome other interface) to an external set of resources (e.g., memory, aworkstation that runs software, etc.). In some embodiments, thecontroller 615 receives data from outside of the IC, formulates the datapackets based on the received data, and routes the data packets to thetiles of the tile array 610 over the bus 605. In some embodiments, thecontroller 615 receives

data from within the IC, formulates the data packets based on thereceived data, and routes the data packets to the tiles of the tilearray 610 over the bus 605.

The data packet is routed through multiple tiles, and passes out of thetop tiles into the transport network 692. In some embodiments, thetransport network 692 is an example of the transport network 215described above by reference to FIG. 2. Additionally, each of theconfigurable tiles includes one or more pipeline registers 665 thatbuffer the signals passing through the bus 605 of the configuration andmonitoring network. Specifically, these pipeline registers 665 act toimprove the timing of the data transport, so that high operatingfrequencies can be achieved. The tiles at the bottom of the tile array610 of FIG. 6 each have two pipeline registers 665, one of which is forpassing signals “up” a column, while another is for passing signals“across” a column. Because of these pipeline registers 665, theconfiguration and monitoring network is fully “pipelined.” In otherwords, more than one set of data can be present within the configurationand monitoring network at any given time by virtue of these pipelineregisters 665.

Each tile also includes a set of decoders 690. The set of decoders 690includes a tile selector that evaluates each packet received through thedata bus of the configuration and monitoring network and determines,based on the contents of the packet (i.e., the opcode and operand)whether that packet was addressed for that tile. The set of decoders 690also includes first and second decoders that determine, based on thecontents of the packet, which resources within the tile are addressed,and the operation specified by the packet to perform at the addressedresources (e.g., read, write, etc.).

The merge tiles 650 of the transport network 692 route the data to andfrom the primary circuit structure along bitlines 652 to the tracebuffer 660 and the deskew and reconstruction circuits 670. In FIG. 6,and in some other figures of this specification, data lines arerepresented with a slash through them and the letter “n” (or a number)next to the slash. These symbols indicate that the line representsmultiple data lines, but is represented as one line rather than renderthe figure difficult to understand by having a separate line for eachbit of width of the line. It will be clear to those of ordinary skill inthe art that: 1) other values of n can be used in other embodiments, and2) multiple instances of “slash n” in a particular figure do notnecessarily represent the same width as each other even within thatparticular figure. Furthermore, when the text or context indicates thata line without a “slash n” is a multiple line bus, the absence of the“slash n” should not be taken to mean that a line is a single bit dataline.

In some embodiments, the primary circuit structure has a known latencythrough each of the tiles of the tile array 610. However, the topologyis constructed such that two signals that pass through different numbersof tiles will take the same amount of time to travel through thetransport network 692. Furthermore, the amount of time it takes for asignal to pass through a set of tiles can be predicted from the paththrough the tiles.

Whilst the time for a round-trip is constant, the transport time from agiven tile to the controller 615 will vary depending on the physicalposition of the tile on the fabric. This raises the issue of how tocompare data that comes from different parts of the configurable IC(e.g., different tiles in the tile array 610). The deskew circuitry 670compensates for the variance in delays caused by bits arriving fromdifferent physical locations. In some embodiments, the deskew circuitry670 also compensates for other delays. Other delays include those causedby retiming of the configured circuit. The deskewing operation of thedeskew circuitry 670 allows the trigger circuits 680 to operate on datathat is adjusted to appear properly simultaneous. In some embodiments,circuitry of the configuration and monitoring network thus performs amask and merge operation, as further described below, such that the datapassing through the configuration and monitoring network is notdisjointed.

In some embodiments, the bandwidth (i.e., the amount of data during agiven time) that the bus 605 can carry to the transport network 692 islimited by the width of the bus 605. In some circumstances, it isdesirable to collect more data bits from a given column than the widthof the bus in that column would allow. In some embodiments, this problemis solved by using the routing fabric of the tiles to send theadditional data bits to tiles in one or more other columns. In otherwords, if the demand from a particular column is higher than thecapacity in that column, then the routing fabric can redirect the signalto another column with excess capacity (i.e., a set of configurablecircuits that are not assigned to the user design). Examples of routingfabric, such

as wiring and interconnects that connect the configurable logic circuitsare disclosed in U.S. Pat. No. 7,295,037, issued Nov. 13, 2007.Moreover, a more detailed discussion for the various componentsillustrated in FIG. 6 and for other components of the primary datastructure and the configuration and monitoring network described hereinis provided for in U.S. Pat. No. 8,069,425, issued on Nov. 29, 2011, andU.S. Pat. No. 7,375,550, issued May 20, 2008. U.S. Pat. No. 7,295,037,U.S. Pat. No. 8,069,425, and U.S. Pat. No. 7,375,550 are incorporatedherein by reference.

In this specification, the figures show the data flowing “up” theconfiguration and monitoring network, then along the transport network692 from left to right, then into a trace buffer 660 to the right of thetransport network 692 and into trigger circuits 680 below the transportnetwork 692. However, it will be clear to one of ordinary skill in theart that other orientations of components other than the particularorientations illustrated are possible within the scope of the invention.For example, the primary circuit structure might send data “down” to atransport network 692 below the tile array, or data might flow from“right” to “left” to reach trigger circuits and/or trace buffers on theleft instead of the right, etc.

As mentioned above, the controller 615 includes an interface to theprimary circuit structure of the IC. In some embodiments, such aninterface is provided through a fabric port. In some embodiments, afabric port provides an interface between the controller 615 of theconfiguration and monitoring network and the primary circuit structure(which performs the “user design”). Thus, the fabric port provides amechanism for the user design to access and control resources of theconfiguration and monitoring network (e.g., configuration bits withinthe configuration and monitoring network). Through the fabric port, theprimary circuit structure is able to interact with the configuration andmonitoring network in an internal manner that is similar to externalmechanisms (e.g., external software communicating with the configurationand monitoring network through a JTAG or some other interface).

A. Deskewing and Reconstruction Units

Physical signals in the fabric must be aligned before they are conveyedto the trigger unit. The signals arrive out of alignment because signalsthat originate from different physical locations on the chip passthrough different numbers of pipelining registers along the way.Assuming that user signals on the user circuit are sourced and sampledat the same fabric clock cycle, the deskew unit corrects any skew causedby signals travelling distances and/or software retiming and routingchoices such that the user signals are realigned again at the inputs ofthe trigger unit.

FIG. 7 conceptually illustrates a group of the configuration andmonitoring network units of some embodiments of the invention. As shown,the configuration and monitoring network units include trace buffer 705,trigger unit 710, deskew and reconstruction units 715 for trace buffer705 and deskew and reconstruction units 720 for trigger unit 710.

Each deskew unit includes many deskew lines, each providing a 1-bitwide, programmable delay that can be used to align signals thatparticipate in a user-defined trigger. In some embodiments, there are atleast as many deskew lines as there are trigger inputs. In the exampleof FIG. 7, there are 128 1-bit lines 740. Additional deskew lines areprovided in some embodiments to be used as trace buffer inputs. Thedeskew units are controlled by the values of the configuration bits 750.The deskew units in some embodiments are controlled from static bits725. The configuration-related logic runs on the configuration clock(cfg_clk) domain.

FIG. 8 conceptually illustrates a group of the configuration andmonitoring network units of some embodiments of the invention in moredetail. The deskew units in the example of FIG. 8 include three stripesof deskew units 805-815. Each stripe has deskew units for both the tracebuffer 820 and the trigger unit 825. The trigger unit 825 fires atrigger signal 835 when the trigger unit identifies a data value, a setof values, or a sequence of values coming in that satisfy one or moreuser specified conditions set of the incoming data. In the illustratedexample, the firing of the trigger signal causes the trace buffer 820 tostore and record data that is being streamed out from the merge tiles840 through the pipe and FIFO unit 845. In other embodiments, the firingof the trigger cause other actions such as storing data in storage otherthan the trace buffer (e.g., in RAM), sending data out of the IC,sending the data into the primary circuits of the IC, causing one ormore reconfigurable circuits to reconfigure, halting the IC, etc.

The deskew units data path 850 operates on the full-rate sub-cycle clock(sread_clk). This clock is throttled with enable signals to createsupport for various clocking modes and to extend the delay capacity ofthe deskew units. Data in a deskew line is pushed through multiplepipelining stages at throughputs of one bit per fabric clock cycle, butis still synchronous to the sub-cycle clock. The fabric clock is avirtual clock; its frequency is the subcycle clock frequency divided bythe looper (e.g., divided by 4 in a 4-loopered scheme). The distinctionbetween the fabric clock and user clock is needed in the presence ofrescaling. The user clock is what the user design defines in the RTL.Once the design is mapped to spacetime, it may be rescaled and otherwisetransformed to a fabric clock.

B. Different Configurations for Reconstruction Units

Some embodiments utilize the user signals that were expressed in theuser RTL in order to build conditions that fire the trigger signal.However, user signals as originally expressed in the RTL source code maynot be available on-chip for the following reasons: (1) synthesisoptimization—user signals may have been eliminated during synthesis, orthey may be contained within a LUT and (2) rescaling—multiplefabric-cycle rate signals may need to be recombined to create a singleuser-cycle rate user signal.

Some embodiments employ two different strategies to create signals thatcan be used for triggering: (1) user signals that were eliminated duringsynthesis are reconstructed by logically combining other existent fabricsignals with one or more LUTs. The number of component signals needed tocombine to create the RTL signal is denoted n and (2) multiple versionsof a user signal that are created by rescaling are recombined into asingle fabric signal with a rescaling multiplexer (mux).

FIG. 9 conceptually illustrates examples of different configurations905-930 for the basic reconstruction units. The rescaling factor in useby the clock domain under debug is denoted k. The number of componentsignals needed to create the user signal is denoted n. The rescaling mux945 selects one of k rescaled signals, one per sub-cycle. One or moreLUTs 940, and a rescaling mux 945 are installed between the deskew unitand the trigger unit. The LUT can be used to regenerate signals lostthrough optimization and the rescaling mux 945 can create the compositeuser signal by selecting the appropriate k rescaled signal during eachfabric cycle. The configurations 905-930 can also handle simultaneousregeneration via the LUT and composite signal generation via therescaling mux. For example, the reconstruction unit 910 with n=2, k=4can reconstruct signals from two related signals via a 2-LUT, and canrecombine signals for net-lists transformed by rescaling up to 4×. Thedeskew lines 935 align signals as they arrive from the merge network.

FIG. 10 conceptually illustrates the reconstruction circuitry in someembodiments of the invention where the number of component signalsneeded to create the user signal is up to 3 and the rescaling factor is4. The entire block of circuitry 1005 serves a single trigger or tracebuffer input. Three of the four LUTs 1010 and their related deskew unitsare not used if no rescaling is applied. Two of the four LUTs andrelated deskew units are not used if 2×rescaling is applied. Each deskewunit contains up to four 3-input LUTs (3-LUTs). For n from 1 to 3 theLUTs are used independently. For n equal to a larger than the number ofthe inputs to the LUTs (i.e., 4 or more) the LUTs are chained togetherto create wider functions. In some embodiments, the deskew unitimplemented can only handle a certain value of n for a given value of k.In order to support larger values of n, the LUTs of other deskew unitsare borrowed. In the case that the other (donor) deskew unit is notusing the LUT (e.g., k=1, n=2), there is no loss in capacity.

Chaining of multiple deskew lines allows combining of the deskew lineblocks to handle worst-case skew for lower values of loopemess and/orextreme signal locations. Chaining multiple deskew lines together canreduce the number of deskew unit outputs available for triggering, butmakes it possible to achieve longer delays.

FIG. 11 conceptually illustrates chaining of multiple deskew lines insome embodiments of the invention. Signal alignment (S delay) deskewlines 1105 are chained internal to a deskew unit. The chaining happensthrough the merge sample (MSample) units 1110. In order to reduce wirecongestion in the design, not all S Delay blocks have the extra chaininginput, as shown in FIG. 11. At most N−1 inputs to a function would needextra delayed, and not every S input is used for most functions. LUTchaining 1115 enters the merge sample unit, similar to S-delay chaining.The source signals are the chained out outputs from another Deskew Unit.

III. Views

Some embodiments provide a method of creating views for monitoring andcontrolling the operations of an IC. A view is a tool for specifyingevents and response to the events. Each view is a statement thatidentifies what to do (e.g., what data items to collect or what controlsto activate) when a condition becomes true. For instance, a user cancreate a view to start collecting data when a set of user design signalssatisfies a particular condition.

Views enable inspection of production designs in the field or in the labwhile the IC is performing operations of the user design. The use ofviews in some embodiments allows all application-visible states to beunobtrusively monitored and inspected. No recompilation or replacementof the code running on the IC and no pre-declaration of viewable signalsare required.

Some embodiments utilize an interactive process for describing theviews. The result of the process is a text file that describes a viewspecification. Some embodiments use a proprietary format while otherembodiments use a subset of System Verilog to define viewspecifications.

A property is a statement that describes a set of relationships betweensignals in a design. For instance, a counter must not exceed 127, “Ackmust go high between 1 and 3 cycles after receiving a Req”, etc. Anassertion is a construct that describes actions to be taken when aproperty is or is not met. As such, the assertion can be expressed as:

-   -   if (property) then action1 else action2

The following is an example of an assertion named “Ctr”, which generatesa warning if the value of “counter” exceeds 127:

-   -   Ctr: assert (counter<=127) else $warning (“Counter out of        range”)

The following is an example of an assertion named “ReqAck” thatgenerates an error when an Ack signal is not detected:

-   -   ReqAck: assert property (@(posedge Clock) Req 1->##[1:2]Ack)        $display    -   (“Everything's ok”) else $error (Ack not detected”)

A view generally expressed as:

-   -   View1: if(condition) $performAction        can be expressed using a restricted subset of System Verilog as:    -   View1: assert (!condition) else $performAction

Some embodiments utilize the System Verilog “bind” construct toassociate an assertion to a module or an instance in a user design bylooking inside a pre-existing RTL instantiation hierarchy without theneed to modify the original source code. The following is a example ofan RTL source code in some embodiments of the invention:

-   -   //The DUT (original RTL)    -   module dut(input a, input b, input clk);    -   reg c;    -   always @ (posedge clk)begin    -   c<=a&b;    -   end    -   endmodule    -   //Assertion wrapped in a module (written in a different file)    -   module dut_sv_assert_module (input i, inputj, input clk);    -   property prop0; @(posedge clk) i!=j; endproperty: prop0    -   assert_prop0: assert property(prop0) else $traceToBuffer(i,j);    -   endmodule    -   //Binding of assertion to DUT    -   module top;    -   . . .    -   bind dut dut_sv_assert_module dut_sv_assert_instance (.i(a),        .j(b), .clk(clk));    -   endmodule: top

In the above example, the module “DUT” can be created as part of theuser design module and the assertion and the binding can be createdseparately at a different time (before or after the design has closedtiming or shipped) and without changing the original RTL.

In some embodiments, languages other than System Verilog are used tospecify views. The following is an example of a proprietary format forspecifying a view to trace a signal for 201 cycles, centered around thetime of triggering:

-   -   trace buffer a0.b1.bar_reg##[−100:100]

The following is an example of a proprietary format for specifying aview to trigger when an instance of bar_reg==0:

-   -   trigger EQ(a0.b0.bar_reg, 1′b0)

The following is an example of a proprietary format for specifying aview to trigger when a0.foo_reg (at time T+2)==a1.foo_reg (at timeT+3)+2, and call T the time when the trigger happens:

-   -   trigger EQ(a0.foo_reg##[2], AND(a 1.foo_reg##[3], 2′b10))

The following is an example of a proprietary format for specifying aview to count (but not trace) the number of posedges on this signalduring a 3001 elk period (triggers happened at T):

-   -   count POSEDGE a1.b1.mySIO.gadget.out##[−1000:2000]

In some embodiments, the trigger unit has three level of logic and canrepresent expressions such a:

-   -   OR (AND (EQ(a,5), NEQ(b,c)), XOR(POSEDGE(d), NEGEDGE(e)))

As described above, some embodiments perform rescaling which transformsthe original set of circuits into the rescaled set of circuits by makingmultiple copies of the original set of circuits. When a clock domain isrescaled by a factor k, each computation is replicated k times. Thefabric clock (FC) period is k times the original user clock (UC) period.The product of the two is such that the throughput of the clock domainis unchanged.

Assume that the user wants to observe and trigger a 1-bit register “a”,which has been rescaled by k=4 but not optimized. Register “a” will havea sequence of values, expressed as a[i]. During each FC cycle, therewill be four distinct values of “a” present at different x, y, and timecoordinates in the fabric: a_0[j], a_1[j], a_2[.i], a_3[.i], wherea_X[j]=a[i+X].

A user can ask a question such as: “inform me of the first value of isuch that a[i]==1′b1”, which is one of the basic operations needed fortriggering. The four values of “a” over time are replaced with fourdistinct values that simultaneously live on the fabric as a_0, a_1, a_2,a_3. The FC period is four times as long, so the overall throughput ofthe fabric is identical to the throughput of the user design. To answerthe above user inquiry, the trigger unit uses a 1b value, to inform theuser that the condition the user asked for has become true during thethis FC cycle.

Some embodiments use two ways to evaluate the expression a[i]==1′b1 onthe trigger unit when it has been rescaled: (i) in series by sending the4 values a_0 . . . 3 to a 1-bit slice of the trigger unit over a FCperiod (ii) in parallel by sending the 4 values to 4 1-bit slices over afabric cycle period. In some embodiments, the trigger unit allows‘series’ evaluation for k<=4 and for k>4 uses parallel evaluation.

In some embodiments, each view is translated to create an overlay thatprovides configuration data for the configuration and monitoring networkto perform a set of tasks without changing the user design. FIG. 12conceptually illustrates a process 1200 for utilizing an overlay createdfrom a view to perform an action when an event occurs in someembodiments of the invention. As shown, the process loads (at 1205),while the IC is performing user design operations, an overlay thatincludes configuration data for the configuration and monitoring networkto monitor for an event and to take actions after the event istriggered. The overlay corresponds to a particular view that a user hascreated to identify the event and the actions to take after the eventoccurrence.

The process then monitors (at 1210) the signals that trigger the event.The process then determines (at 1215) whether the event conditions aresatisfied. If not, the process proceeds back to 1210 to continuemonitoring the signals. Otherwise, the process performs (at 1220) theactions specified to be taken after the triggered event. The processthen ends.

The views can be used in the lab or in the field. Examples of thedifferent applications and uses of views include in-the-field systemmonitoring (e.g., determining the status of an IC implementingnetworking functions in a data center), in-system performance evaluation(e.g., determining why a system reboots every 30 minutes under real-lifetraffic), debugging of the board and system integration (e.g.,determining why a design that worked in simulation does not work on anactual board), etc.

A. Lab System Debugger

FIG. 13 conceptually illustrates an example of in-the-lab operation modefor an on-board configured IC in some embodiments of the invention. TheIC 1305 includes a set of reconfigurable circuits and a configurationand monitoring network. As shown, the IC 1305 is included on an IC board(or circuit board) 1320. The IC communicates with different on-boardhardware modules such as application-specific integrated circuits(ASICs), etc., 1340. The ASIC is customized for a particular use, ratherthan intended for general-purpose use. The IC 1305 also includesinternal memory (not shown) and/or accesses storage devices such asSRAM, RAM, etc., 1325 on the IC board 1320.

The IC 1305 is connected to a computing device 1310. The computingdevice is, for example, a desktop or a portable computing device thatincludes either a separate or an integrated display 1315. The computingdevice includes software tools to create views, translate views tooverlays, and load created or pre-existing overlays into the IC 1320.

The computing device 1310 sends overlays that include configuration datafor the configuration and monitoring network to the IC while the IC isperforming user design operations. Each overlay corresponds to a viewcreated by a user that specifies an event and the actions to take afterthe event occurs. The IC 1305 receives the overlays as a configurationbitstream 1330. Once an event is triggered, the IC performs thespecified actions and sends the results back to the computing devicethrough a set of communication lines 1335. The computing device 1310analyzes the received data and/or displays the data on the display 1315.

In some embodiments, the communication between the computing device 1310and the IC 1305 is through a dedicated communication module. In otherembodiments (not shown), the communication is through other on-boardhardware such as a controller or a microprocessor. In some embodiments,the tools in the computing device allow a user to select aserializer/deserializer (SerDes) in the IC to dedicate for communicationbetween the computing device and the IC. The tools also provide controlto instantiate the communication. This communication mechanism and thetools are utilized in some of the other configurations described belowand are not repeated for simplicity.

FIG. 14 conceptually illustrates an example of in-the-lab operation modefor an embedded configuration in some embodiments of the invention. Insome embodiments, a group of reconfigurable circuits and a configurationand monitoring network are embedded in an IC (such as an ASIC) to docomputations as well as inspecting the state and monitoring the othercomponents of the ASIC. In some embodiments, these reconfigurablecircuits are not pinned out and are accessible through other componentsof the ASIC. The ASIC is an IC that is customized for a particular use,rather than intended for general-purpose use. An ASIC is eitherpre-manufactured for a special application or is custom manufactured,for example by using components from a “building block” library ofcomponents, for a particular application.

As shown in FIG. 14, a group of reconfigurable circuits withconfiguration and monitoring network 1405 is embedded in an IC such asan ASIC 1415. The ASIC is customized for a particular use, rather thenintended for general-purpose use.

In this example, the ASIC also includes a controller such as amicroprocessor 1410, memory 1435, and other hardware 1430. The ASIC isinstalled on an IC board 1450, which optionally includes other on-boardhardware 1440.

The group of reconfigurable circuits with configuration and monitoringnetwork 1405 participates in operations of the ASIC such as performingmathematical and/or logical operations. In addition, the group ofcircuits 1405 is utilized to receive configuration overlays that includeconfiguration data for the configuration and monitoring network whilethe group of circuits 1405 is performing user design operations.Different internal signals of the ASIC as well as signals received fromthe outside of the ASIC can be connected to the set of reconfigurablecircuits for monitoring and debugging purposes.

The ASIC 1415 is connected to a computing device 1420. The computingdevice is, for example, a desktop or a portable computer that includeseither a separate or an integrated display 1425. In this example, thecomputing device 1420 is connected to the controller 1410 in the ASIC1415. The computing device 1420 sends overlays that includeconfiguration data for the reconfigurable circuits to the ASIC while theASIC is performing user design operations. Each overlay corresponds to aview created by a user that specifies an event and the actions to takeafter the event occurs.

In the example of FIG. 14, the controller 1410 is connected to thereconfigurable circuits 1405. The controller sends the overlays receivedfrom the computing device 1420 to the reconfigurable circuits 1405 as aconfiguration bitstream 1455. Once an event is triggered, thereconfigurable circuits 1405 perform the specified actions and send theresults back to the controller 1410 through a set of communication lines1490. The controller sends the data received from the reconfigurablecircuits 1405 to the computing device 1420. In some embodiments, thecontroller (e.g., a microprocessor) further processes the data and sendsthe processed data to the computing device 1420. The computing device1420 further analyzes the received data and/or displays the data on thedisplay 1425.

FIG. 15 conceptually illustrates a system for generating and using viewsto debug and monitor operations of an IC in some embodiments of theinvention. As shown, a set of electronic design automation (EDA) tools1525 (e.g., synthesis 1530, placement and routing 1535, poweroptimization 1540 tools, etc.) is utilized to configure an IC chip 1550that includes reconfigurable circuits (or the configure thereconfigurable circuits embedded in an ASIC). The user design 1505(e.g., written as design abstractions in a language such as registertransfer language (RTL)) and a set of design constraints 1515 (e.g.,written in a language such as Synopsys design constraints SDC) are givenas input to EDA tools 1525.

The EDA tools generate configuration data 1545 to implement the userdesign. The configuration data 1545 is loaded into the IC chip 1550 toconfigure the configurable circuits of the IC to perform the operationsof the user design. As shown, the information 1570 regarding the userdesign is stored in EDA tool database 1575.

The system of FIG. 15 is also capable of generating and loadingincremental configuration data while the IC is performing the operationsof the user design and without recompiling or reloading the user design.The overlay generation 1580 receives user design information 1570 andviews 1555 (that describe actions to take when a set of user designsignals satisfy a condition) and generates incremental configurationdata 1585. The views 1555 are generated through a user interface 1560such as a graphical user interface or a text editor. The incrementalconfiguration data 1585 adds the functionality specified by the view tothe user design that is running on the IC without disturbing the userdesign, (that is, without replacing the configuration data loaded in theIC to perform the user design), and without stopping the operations ofthe IC when the IC is operating and performing the operations of theuser design.

FIG. 16A illustrates an example of a user interface 1600 for generatingviews in some embodiments of the invention. In some embodiments, inaddition or instead of the user interface 1600, a text editor is used togenerate the views. As shown in FIG. 16A, the user interface includes adesign browser that displays a list 1605 of user design signals. Theuser design signals are the signals specified in the user design. Thesesignals may not correspond to actual signals on the IC fabric due tooptimization and/or rescaling operations. As described above byreference to FIGS. 9-10, the user signals are reconstructed from thesignals on the IC fabric. In some embodiments, each user signals isconceptually represented as a node in a hierarchical tree that shows howthe signal is built from other signals (which are also shown as othernodes of the hierarchical tree) in the user design.

The user interface 1600 may provide a tool 1610 for clock selection. Theuser interface also provides tools 1635-1650 for specifying triggeringevents and a tool 1660 for setting the trigger delays 1655. The signalsfor each trigger specification 1635-1650 are selected from thehierarchical list of signals 1605 and included in the correspondinggroup of signals 1615-1630. For instance, when a user navigates throughthe list of signals 1605 and identifies one of the signals for a view(e.g., by activating a user selection device such as a mouse button), apop up menu is displayed that allows the user to identify the trigger(1635-1650) and the function (e.g. one of the functions for theconditional sentence of the view) to apply to the selected signal. Theselected signal is then displayed in one of the groups 1615-1630 thatcorresponds to the selected trigger. The user then repeats the sameprocess by selecting the next signal for the same or a differenttriggering event. In some embodiments, a view is expressed as aconditional sentence. Views are used. e.g., for assertion and alertchecking such as:

-   -   when (top.tcpflow21.state==RUNNING)        traceToSerDestop.tcpflow21.state

This view indicates that the value of the signal top.tcpflow21.state hasto be monitored and when the value is equal to the predefined valueRUNNING, the values of the signal has to be traced to a SerDestransceiver.

Another example of a view used for assertion and alert checking is:

-   -   when (top.sourceip!=top.watchlist) traceToSerDes top.sourceip

This view indicates that the value of the signal top.sourceip has to bemonitored and when the value is not equal to the value of top.watchlistsignal, the top.source signal has to be traced to SerDes.

Another examples for the use of views is to track packets. For example:

-   -   when (top.sourceip==top.watchedip) traceToDRAM top.packetpayload

This view indicates that the value of top.sourceip (e.g., the packetidentification) has to be monitored and when the value is equal totop.watchedip (e.g., a desired packet identification), top.packetpayload(e.g., the packet payload) has to be stored in DRAM.

-   -   when (top.traffictype==“video”) traceToSerDes top.packetpayload

This view indicates that top.traffictype (e.g., the type of a receivedpacket) has to be monitored and the value indicates “video”,top.packetpayload (e.g., the video stream) has to be traced to SerDes.

Once the user defines an event by selecting the signals to trigger theevent and to trace the data, the user selects a control 1665 to generatean overlay and to load the overlay into the part (e.g., into the IC 1305in FIG. 13 or the group of reconfigurable circuits 1405 in FIG. 14).Once the results are received from the part, a waveform viewer islaunched to show the received data in real-time. Some embodiments runoverlays on the part one at a time.

FIG. 16B illustrates another example of a user interface 1670 forgenerating views in some embodiments of the invention. As shown, theuser interface includes a signal browser display area 1675 that displaysa list of user design signals. The user design signals are the signalsspecified in the user design. These signals may not correspond to actualsignals on the IC fabric due to optimization and/or rescalingoperations. In some embodiments, each user signals is conceptuallyrepresented as a node in a hierarchical tree that shows how the signalis built from other signals (which are also shown as other nodes of thehierarchical tree) in the user design. As described above by referenceto FIGS. 9-10, the user signals are reconstructed from the signals onthe IC fabric.

The user Interface 1670 also includes a display area 1680 for displayingthe list of selected signals for views. The user interface also includesa display area 1685 for displaying a list of selected trigger signals.The information in display areas 1680 and 1685 can be scrolled up anddown by using the scrolling controls 1682 and 1687, respectively.

The user interface also provides a list 1695 of control buttons that areselectable or greyed out as necessary. In the example of FIG. 16B, thecontrol buttons include a control 1697 to compile views to createincremental configuration data, a control 1698 to download theincremental configuration data into the IC (e.g., while the IC isrunning), and a control 1699 to start communicating with the IC.

FIG. 17 conceptually illustrates a waveform viewer 1700 launched todisplay waveform representation of signals being traced after an eventspecified in a view is triggered in some embodiments of the invention.The waveform viewer 1700 in some embodiments is displayed as a window inthe user interface 1600 of FIG. 16A or the user interface 1670 of FIG.16B. In other embodiments, the waveform viewer is launched as a separatewindow.

As shown, the waveform viewer includes a display area 1715 fordisplaying signal names 1710 that were identified in the view to betraced after an event occurs. The waveform viewer includes anotherdisplay area 1720 to display the waveforms 1705 associated with thetraced signals. In some embodiments, the data received from the part isconverted to an ASCII-based value change dump (VCD) format and passed tothe viewer. In other embodiments, the waveform viewer uses a differentdata format to display the signal waveforms.

The user interfaces of FIGS. 16A, 16B, and 17 allow graphicalinteractive selection of user design signals, incremental generation anddownload of bitstream, receiving of data after events are triggered, anddisplay of the results. Data received from the IC after an event istriggered is translated into user design signals if necessary prior todisplaying the results.

FIG. 18 conceptually illustrates a process 1800 for using views in a labenvironment in some embodiments of the invention. As shown, the processloads (at 1805) configuration data for a set of reconfigurable circuitsof the IC to perform operations of a user design. The set ofreconfigurable circuits are, instance, a part of the IC 1305 shown inFIG. 13 or the reconfigurable circuits 1405 embedded in the ASIC 1415shown in FIG. 14.

The process then operates (at 1810) the IC using the loadedconfiguration data. The process then receives (at 1815) a view thatdefines a set of condition for a set of user design signals to triggeran event for taking a set of actions. For instance, the process receivesa view when a user uses the user interface 1600 or uses an editor todefine a view.

The process then translates (at 1820) the user design signals specifiedin the view to actual signals on the IC. For instance, the processtranslates the user signals to actual signals using the reconstructioncircuits defined by reference to FIGS. 9 and 10, above. The process thengenerates (at 1825) an incremental overlay to define additionalconfiguration data to allow, without recompiling the code that isrunning on the IC, monitoring of signals for the event and to take thespecified actions after the event is triggered.

Next, while the IC is performing user design operations and withoutstopping the operations of the IC or replacing the configuration datathat is loaded in the IC, the process loads (at 1830) the incrementaloverlay that includes the configuration data into the IC through theconfiguration and monitoring network. The process then monitors (at1835) the signals that trigger the event while the IC is stillperforming the operations of the user design based on the configurationdata loaded in the IC in operation 1805.

The process then determines (at 1840) whether the event conditions aresatisfied. If not, the process proceeds back to 1835 to continuemonitoring the signals. Otherwise, the process performs (at 1845) theactions specified in the view while the IC is still performing theoperations of the user design. The process then ends.

B. In-the-Field Operation Mode

FIG. 19 conceptually illustrates a localized in-the-field configurationof an IC in some embodiments of the invention. As shown, an on-boardcontroller such as a microprocessor 1910 and an IC 1905 withconfiguration and monitoring network are included on an IC board 1925.Other components of the IC board (if any) are not shown for simplicity.

The on-board controller 1910 stores (or retrieves from onboard memory)different overlays corresponding to different views. The on-boardcontroller receives data from the IC 1905 through a set of communicationlines 1920. Depending on the data received, the controller loads anincremental configuration bitstream 1915 corresponding to one of thestored overlays into the IC. The on-board controller in some embodimentsis programmed to load different overlays based on the data received fromthe IC 1905. The on-board controller stores a set of overlays andautonomously determines which overlay (if any) to load into the ICwithout any interaction with outside the IC board.

FIG. 20A conceptually illustrates an extended in-the-field configurationof an IC in some embodiments of the invention. As shown, the IC board2025 in FIG. 20A has similar configuration as the IC board 1925 in FIG.19, except the on-board controller 2010 in FIG. 20A also communicateswith a control center 2035 through a network 2030 such as the Internet.The control center is, for instance, a computing device with a userinterface. The on-board controller 2010 sends the data received from theIC to the control center 2035 and receives overlays with incrementalconfiguration data to load into the IC 2005.

FIG. 20B conceptually illustrates an extended in-the-field configurationof an IC in some embodiments of the invention. As shown in FIG. 20B, theIC 2005 on the IC board 2025 communicates with the control center 2035through the network 2030 such as the Internet without going through anon-board controller. The IC 2005 sends data to the control center 2035through communication line 2020 and receives overlays as bitstreams 2015to load.

FIG. 21 conceptually illustrates a localized in-the-field configurationof an embedded configuration of some embodiments of the invention. Asshown, a group of reconfigurable circuits with a configuration andmonitoring network 2105 is embedded in an IC such as an ASIC 2115. Inthis example, the ASIC also includes a controller such as amicroprocessor 2110, memory 2135, and other hardware 2130. The ASIC isinstalled on an IC board 2150, which optionally includes other on-boardhardware 2140-2145.

The controller 2110 stores (or retrieves from memory 2135) differentoverlays corresponding to different views. The controller receives datafrom the group of reconfigurable circuits 2105 through a set ofcommunication lines 2120. Depending on the data received, the controllerloads an incremental configuration bitstream 2155 corresponding to oneof the stored overlays. The controller 2110 in some embodiments isprogrammed to load different overlays based on the data received fromthe group of reconfigurable circuits 2105. The controller 2110 stores aset of overlays and autonomously determines which overlay (if any) toload into the IC without any interaction with outside the IC board.Different internal signals of the ASIC as well as signals received fromthe outside of the ASIC can be connected to the set of reconfigurablecircuits for monitoring and debugging purposes.

In some embodiments, another on-board hardware 2145 acts as a controllerto store the overlays to load into the group of reconfigurable circuits.In these embodiments, the on-board controller 2145 (e.g., amicroprocessor) stores the overlays and sends them to the controller2110 in the ASIC in order to load into the group of reconfigurablecircuits 2105. The on-board controller 2145 receives data from the groupof reconfigurable circuits 2105 through the controller 2110. Dependingon the data received, the controller 2145 sends a configurationbitstream corresponding to one of the stored overlays to the ASIC toload into the group of reconfigurable circuits 2105.

FIG. 22 conceptually illustrates an extended in-the-field configurationof an embedded configuration of some embodiments of the invention. Asshown, the IC board 2250 in FIG. 22 has similar configuration as the ICboard 2150 in FIG. 21, except the controller 2210 in FIG. 22 alsocommunicates with a control center 2265 through a network 2260 such asthe Internet. The control center is, for instance, a computing devicewith a user interface. The controller 2210 sends the data received fromthe IC to the control center 2265 and receives overlays to load into thegroup of reconfigurable circuits 2205.

FIG. 35 conceptually illustrates an ASIC with a configurable device insome embodiments of the invention. In the example of FIG. 35, the ASIC3500 includes several custom-made processing units 3505-3515 andEthernet interfaces 3520-3530. Processing unit 3505 receives data frominput line 3535, processes the data, and sends output data throughoutput line 3540 to processing unit 3515 using the 10 GB Ethernetinterface 3520. Similarly, processing unit 3510 receives data from inputline 3545, processes the data, and sends output data through output line3550 to processing unit 3515 using the 10 GB Ethernet interface 3525.Processing unit 3515 processes the received data and outputs data on theoutput line 3555 using the 20 Gb Ethernet interface 3530.

As shown, the ASIC also includes a configurable device 3560. Forexample, the configurable device includes a group of reconfigurablecircuits and a configuration and monitoring network as described byreference to FIGS. 14, 21, and 22, above. The configurable device isembedded in the ASIC. In this example, the reconfigurable circuits ofthe configurable device 3560 are not pinned out and are accessiblethrough other components of the ASIC.

The configurable device includes a group of circuits 3565 that monitorsinput data lines 3535 and 3545, a group of circuits 3570 that monitorsoutput lines 3540 and 3550 of the 10 Gb Ethernet interfaces, a group ofcircuits that monitors output line 3555 of the 20 Gb Ethernet interface,and a group of circuits 3580 that checks data generated by configurabledevice 3580. As described by reference to FIGS. 14, 21, and 22,configurable device receives incremental configuration datacorresponding to different views from and provides data to othercomponents of the ASIC (not shown for simplicity). The ASIC can bedesigned in a way that any data line that needs to be monitored can beconnected to the configurable device. The configurable device can thenbe programmed during the operation of the ASIC to receive incrementalconfiguration data corresponding to different views without disturbingthe operations of the ASIC, monitor the specified signals, and takeaction once a triggering event occurs. In addition to monitoring anddebugging operations, the reconfigurable circuits of the embeddedconfigurable device 3560 can receive configuration data sets andconfigurably perform operations of a user design based on the receivedconfiguration data sets.

FIG. 23 conceptually illustrates a process 2300 for operating an IC inthe field in some embodiments of the invention. As shown, the processoperates (at 2305) the IC by loading a set of configuration data intothe IC. The IC in some embodiments is an IC with a set of reconfigurablecircuits and a configuration and monitoring network such as ICs 1905 and2005 shown in FIGS. 19, 20A, and 20B. The IC in other embodiments is anIC such as ASICs 2115, 2215, and 3500 shown in FIGS. 21, 22, and 35,respectively. These ICs include a group of reconfigurable circuits 2105,2205, and 3560 with configuration and monitoring network. In someembodiments, the configuration data is in permanent memory such as flashmemory in the same IC board and is loaded into the IC upon power up.

Process 2300 then receives (at 2310) an overlay with an additionalconfiguration data to allow, without disturbing the user design that isrunning on the IC, monitoring of signals for the event and to take thespecified actions after the event is triggered. The overlay is generatedbased on a user-defined view. The process receives the overlays eitherfrom on-board controllers (such as on-board controllers 1910 or 2010 inFIGS. 19 and 20A, respectively), in-the-chip controllers (such ascontroller 2110 in ASIC 2115 in FIG. 21 or controller 2210 in ASIC 2215in FIG. 22), or a remote control center (such as control center 2035 inFIG. 20B).

Next, while the IC is performing user design operations and withoutstopping the operations of the IC, the process loads (at 2315) theoverlay that includes the configuration data into the IC through theconfiguration and monitoring network. The process then monitors (at2320) the signals that trigger the event. The process then determines(at 2325) whether the event conditions are satisfied. If not, theprocess proceeds back to 2320 to continue monitoring the signals.Otherwise, the process performs (at 2330) the actions that correspond tothe triggered event while the IC is still performing the operations ofthe user design. The process then ends.

C. Triggering Events and Taking Actions Using Views

FIG. 24 conceptually illustrates a process 2400 for triggering eventsand taking actions defined by a view in some embodiments of theinvention. The process loads (at 2405), while the IC is performing userdesign operations, an overlay that includes configuration data for theconfiguration and monitoring network to monitor for an event and to takeactions after the event is triggered. The overlay has been generatedfrom a user-defined view that specified the conditions to trigger theevent and the actions to take after the event is triggered.

The process then monitors (at 2410) the signals that trigger the event.The process then determines (at 2415) whether the event conditions aremet. If not, the process proceeds back to 2410 to continue monitoringthe signals. Otherwise, when the trigger conditions are met, the processdetermines (at 2420) whether the action to take is collecting andstoring of data. If yes, the process collects data and saves (at 2425)the data in storage such as trace buffer, DRAM, etc., as specified forthe action. The process then ends.

Otherwise, the process determines (at 2430) whether the action to takeis to send data back into the IC. If yes, the process (at 2435) sendsthe identified data to the destination in the IC as defined by theaction. For instance the process sends a predetermined value or thesampled value of a signal to a particular register or a particular inputof a component in the IC. The process then ends.

Otherwise, the process determines (at 2440) whether the action to takeis to send data out of the IC to another on-board hardware. If yes, theprocess sends (at 2445) the data to the on-board hardware identified bythe action. For instance the process uses a SerDes to send theidentified data to an identified port in the IC. The process then ends.

Otherwise, the process determines (at 2450) whether the action is tosend data out of the IC board. If yes, the process sends (at 2455) thedata to the remote destination identified by the action. For instancethe process uses SerDes to send the identified data to a receiverprocess outside the IC board. The process then ends.

Otherwise, the process determines (at 2460) whether the action is toreconfigure a portion of the configurable circuits in the IC or to haltthe IC. If not, the process ends.

Otherwise, the process loads (at 2465) an overlay that includes therequired configuration data for the portion of the configurable circuitsthrough the reconfiguration network (or if the action is to halt the IC,the process triggers a signal to halt the IC). The process then ends.

IV. Examples of the Use of Views and Overlays

A. Root Fault Cause Analysis

FIG. 25 conceptually illustrates a process 2500 for performing rootcause analysis in some embodiments of the invention. Process 2500 isperformed by an IC that includes reconfigurable circuits and aconfiguration and monitoring network in some embodiments. As shown, theprocess loads (at 2505), while the IC is performing user designoperations, an overlay that includes configuration data for theconfiguration and monitoring network to monitor for an event based on aset of internal and/or input signals and to take actions after the eventis triggered.

The process then monitors (at 2510) the signals that trigger the event.The process stores (at 2515) the values of the monitored signals and/orthe value of additional signals to be examined after the event istriggered. The process then determines (at 2520) whether the eventconditions are met. If not, the process proceeds to 2510 to continuemonitor the signals and store the specified values.

Otherwise, the process performs (at 2525) the actions associated withthe event such as sending the data identified by the action outside theIC. While the IC is performing user design operations and withoutstopping the operations of the IC, the process then iteratively (i)receives (at 2530) overlays that include incremental configuration dataand (ii) sends (at 2530) the stored signal values identified in eachoverlay outside the IC. The process then continues performing (at 2535)user design operations.

FIG. 26 conceptually illustrates a process 2600 for performing rootcause analysis in some embodiments of the invention. Process 2600 isperformed by a controller outside the IC in some embodiments. As shown,the process receives (at 2605) an indication from the monitored IC thatan event related to an assertion has occurred. The process thenidentifies (at 2610) the set of signals that caused the event to occur.A simple example of root cause analysis is the following. When anincorrect value appears at the output “f” of the logic operation“f=a&b”, the next step is to check if either “a” or “b” is at fault, andkeep going back (if a and b are results of other logic operations) untilwhat is considered to be the root of the problem is found.

The process then identifies (at 2615) a signal in the set of signalsthat caused the event to occur. The process then determines (at 2620)whether the identified signal is the root cause of the fault thatresulted in the event to occur. If yes, the process has identified theroot cause of the fault. The process takes (at 2645) further actionsbased on the identified root cause of the fault. The process then ends.Otherwise, the process determines (at 2625) whether there are additionalsignals that could cause the identified signal to trigger the event. Ifnot, the process cannot identify the root cause of the current fault.The process optionally loads (at 2650) another overlay to monitor forfurther events. The process then ends.

Otherwise, while the IC is performing user design operations, theprocess sends (at 2630) an incremental overlay to the IC to identifyfurther stored values of the additional signals. The process thenreceives (at 2635) the identified stored values. The process thendetermines (at 2640) whether the root cause of the fault can bedetermined from the value of the received signals. If yes, the processproceeds to 2645, which was described above. Otherwise, the processproceeds to 2625 to continue the attempt to identify the root cause ofthe fault.

B. In-the-Field Fault Injection

Fault injection is a technique regularly used to test how complexsystems, such as ICs, react to unforeseen circumstances such asunexpected or out-of-range inputs, or physical failures of thecomponents, for instance due to normal wear and tear.

FIG. 27 conceptually illustrates a process 2700 for performingin-the-field fault injection in some embodiments of the invention. Asshown, the process identifies (at 2705) a particular user design statewhich is considered to be faulty. The process then identifies (at 2710)the values of a set of signals at the particular IC state.

The process then loads (at 2715), while the IC is performing user designoperations, an overlay that includes incremental configuration data forthe reconfigurable circuits through the configuration and monitoringnetwork to set the values of the set of signals to the identifiedvalues. For instance, when the IC includes a counter that reaches aparticular value only when a fault occurs in the IC, the overlay causesthe particular value to be loaded in the counter without waiting for thefault to actually occur.

The process then loads (at 2720), while the IC is performing user designoperations, an overlay that includes incremental configuration data tomonitor for an event and to take actions after the event is triggered(e.g., to further analyze the fault condition). The process thenmonitors (at 2725) the signals that trigger the event. The process thendetermines (at 2730) whether the event conditions are satisfied. If not,the process proceeds back to 2725 to continue monitoring the signals.Otherwise, the process performs (at 2735) the actions that correspond tothe triggered event while the IC is still performing the operations ofthe user design. The process then ends.

C. Inspection and Search of Networking Packets

Some embodiments utilize views to inspect and search networking packets.For instance, an IC that includes reconfigurable circuits and aconfiguration and monitoring network can monitor packets received at theIC, examine the content, and take actions when a particular packet (e.g.a packet with a particular header) has arrived.

Such an IC can be used to monitor network traffic to determine whetheror not any particular packet has arrived at a destination. For instance,in a data center with thousands of devices such as bridges, switches,routers and gateways, such an IC can be included in each device tounobtrusively monitor the traffic in the data center without stoppingthe operations of the data center.

FIG. 28 conceptually illustrates a process 2800 for packet monitoring msome embodiments of the invention. The process loads (at 2805), whilethe IC is performing user design operations, an overlay that includesconfiguration data to monitor for a particular packet on a specifieddata line. The overlay is generated from a user-defined view. Theprocess then monitors (at 2810) for the particular packet on thespecified data line. This is achieved by programming the trigger unitsuch that, for instance, information that identifies a specified senderis found in the packet header.

The process then determines (at 2815) whether the packet has arrived. Ifnot, the process proceeds back to 2810 to continue monitoring for theparticular packet. Otherwise, the process indicates (at 2820) that thepacket has arrived. For instance, the process raises a flag by setting aparticular location in the memory, by storing data in trace buffer, orby sending a signal to an output line to indicate that the packet hasarrived.

The process then performs (at 2825) other actions indicated by theoverlay such as storing all or a portion of the packet, performingfollow up actions, etc. the process then ends.

FIG. 29 conceptually illustrates a data center 2900 in some embodimentsof the invention. The data center 2900 includes electronic devices2905-2920 such as bridges, switches, routers, gateways, computersystems, storage systems, telecommunication systems, security devices,etc. Only a few electronic devices are shown in FIG. 29 for brevity. Thedata center can be used by a carrier's telecommunication network, by anecommerce website, by a search engine website, by a web portal, etc.

Some of the electronic devices in a data center can include IC's withreconfigurable circuits and a configuration and monitoring network tofacilitate monitoring of the network traffic. Other components of theelectronic devices are not shown for simplicity.

FIG. 29 shows different examples for including such IC's in theseelectronic devices. For instance, electronic device 2915 includes an ICboard 2925 with an IC 2930 and a controller 2935 as described byreference to FIG. 20A, above. The electronic device 2910 includes an ICboard 2940 with an ASIC 2945 with an embedded group of reconfigurablecircuits 2950 and their associated configuration and monitoring networkas described by reference to FIG. 22, above. The electronic device 2920includes an IC board 2955 with an IC 2960 that communicates with theremote control center 2985 through the network 2965 as described byreference to FIG. 20B, above.

Including these ICs in each of the electronic device 2910-2920 allowsthe network traffic coming into and going out of each device to bemonitored and reported to a remote control center 2985 through a network2965 such as the Internet. The remote control center provides facilitiessuch as user interfaces to display data received from the ICs embeddedin the electronic devices, select predefined overlays, create views togenerate new overlays, and send the overlays to the ICs to monitor forfurther packets, to monitor for different events, and to take actionswhen a packet arrives or an event condition becomes true.

In addition, some of the electronic devices (such as electronic device2905) can include an IC board 2970 with an IC 2975 and a controller 2980with a localized configuration described by reference to FIG. 19 or 21,above. In these configurations, the controller 2980 receives data fromthe IC 2975 and loads overlays with incremental configuration data intothe IC based on a pre-programmed scheme without outside (e.g., human)interactions.

FIG. 30 conceptually illustrates tracking of packets between source anddestination nodes in a network according to some embodiments of theinvention. As shown, two devices 3005 and 3010 are connected through aset of network equipment 3020-3045 such as bridges, switches, routers,etc. The devices and the network equipment can be part of a data center3015 or a communication network and include an IC with a group ofreconfigurable circuits and a configuration and monitoring network,e.g., as described by reference to FIGS. 20A, 20B, and 22.

The devices in a network can be connected through many different paths.For instance, devices 3005 and 3010 can be connected through (1) networkequipment 3020, 3025, and 3030, (2) network equipment 3020, 3035, and3030, (3) network equipment 3035, 3040, 3045, etc.

Having multiple paths between a source and a destination node in anetwork can create a loop. When a loop occurs, the network topologypermits a packet to traverse the same network equipment more than once.The loop creates broadcast radiation as the network equipment repeatedlybroadcast packets and flooding the network. The devices in a networkoften perform a spanning-tree protocol to prevent loops from beingformed. Spanning-Tree Protocol exchanges messages between the networknodes to detect loops and remove them by shutting down redundant paths.This algorithm guarantees that there is one and only one active pathbetween two network nodes, and that each network node does not connectback to itself. The devices perform the spanning tree protocol todetermine a spanning tree where each node lies in a path in the networkbut no loops are formed.

In order to perform spanning tree protocol, the network nodes (such asdevices and network equipment 3005-3045) exchange packets among eachother and determine whether a packet arrives at a certain node once,multiple times, or does not arrive after a certain time. In the exampleof FIG. 30, a packet is sent from device 3005 to device 3010 by anintended path through network equipment 3035, 3040, and 3045. The remotecontrol center 3050 sends an overlay 3055 to the devices and networkequipment 3005-3045 through a network. For simplicity the network is notshown and only a few of the overlays are shown. The overlays are createdbased on a view that triggers an event when a particular packet (e.g., apacker with a particular header) arrives and sends data back to remotecenter to indicate that the packet has arrived.

FIG. 31 conceptually illustrates the network of FIG. 30 where a packethas not arrived at an intended destination. As shown, source device 3005has sent data 3105 to the remote control center to indicate that aparticular packet has departed from the device. Also, network equipment3035 and 3040 have sent data to the remote control center to indicatethat the arrival and departure of the packet. Network equipment 3045 anddevice 3010 have not sent any data to indicate the arrival or departureof the packet. After a predetermined timeout, the remote control centerconcludes that the packet has been lost between network equipment 3040and 3045.

Similarly, other network equipment such as 3020-3030 may send dataindicating that they have also received the package. Network equipment3035-3040 may send data to indicate that they have received multiplecopies of the packets. The network devices and equipment collectivelyuse the information about the arrival and departure of packets todetermine the loops and lost links in order to remove redundant pathsbetween the nodes, to ensure that there is at least one path to eachnode in the network and to ensure that no loops exist.

D. Delivery of Enhanced Diagnostics to End Users

Utilizing ICs with reconfigurable circuits and a configuration andmonitoring network allows device manufactures to incorporates such ICsinto their devices and provide a set of monitoring tool to the end usersto monitor their systems. FIG. 32 conceptually illustrates the deliveryof enhanced monitoring and diagnostic tools from device manufacturers tothe end users in some embodiments of the invention. As shown, the ICmanufacturer 3230 manufactures ICs 3205 with reconfigurable circuits andconfiguration and monitoring networks. The IC manufacturer also providestools and user interfaces 3210 to generate views and overlays.

The device manufacturer 3235 receives the ICs and the tools from the ICmanufacturer and incorporates the ICs 3205 in the manufactured devices3220. The device manufacturer creates different views using the tools3210 to trigger events based on different conditions and to takesubsequent actions after an event is triggered. For instance, the IC isembedded in the device to perform mathematical and logical operationsintended by the device and in addition to monitor certain signals in thedevice, trace data, and send the data out to a certain port to bedisplayed on a monitoring console 3215.

The device manufacturer sells the devices 3220 to end users 3240 andprovides the end user with instructions to use the monitoring console3215 to monitor the data received from the ICs embedded in theelectronic devices 3220 and to load overlays that include bitstreamswith incremental configuration data 3225 to implement enhancedmonitoring and diagnostics features.

In addition, when a new problem is reported in the field by the enduser, the device manufacturer can generate more views, create thecorresponding bitstreams to implement the views and send the overlaypackets containing the bitstreams to the end user to load into thedevice in the field. Loading of the overlays, monitoring for the events,subsequent actions and data reporting can be done without hating thedevice from performing its intended operations. For instance, the devicemanufacturer 3235 could be a networking equipment manufacturer and theend user 3240 could be an electronic commerce (ecommerce) company, asearch engine company, a web portal company, or an individual whopurchased the device for personal use.

As an example, consider that the electronic device is the networkingequipment used in a data center. The operations of the networkingequipment can be monitored without interrupting the operations of thedata center. In this example, the IC manufacturer provides the ICs andtools to the device manufacturer to facilitate implementation ofenhanced monitoring and diagnostics features to the end users, who arethe customers of the customers of the IC manufacturer.

E. Executing the Same Assertions During Simulation and Runtime

Today, part of the process for designing microchips is to write code andto write test bench simulations. The designers write the code and makestatement about properties in the code. For instance, a count for theday of the month can never exceed 31. In order to verify the design inthe lab, the designer writes simulation code that checks the value ofthe count for the day of the month.

Writing a condition (or assertion) that the value for day of the monthcan never exceed 31 continuously checks the value during simulation andverification. Usually, when the system is deployed in the field, thesimulation code that checks for the value is not included in the codethat is deployed in the field (e.g., because of the amount of resourcesto be included in the IC to continuously check the veracity of thecondition). Any attempt to check for the value in the field or to checkfor any other values in the field requires recompiling of the deployeduser design, halting the operations of the system in the field, andreloading of the code into the chip. Similarly, to remove the extra codethat checks for the values in the field requires another compilation,halting the operations of the system, and loading of new code into thechip.

Using the disclosed ICs that include reconfigurable circuits and aconfiguration and monitoring network allows loading overlays thatprovide configuration bitstreams to the chip without halting theoperations of the chip or recompiling and replacing the code that isoperating on the chip. The same assertions used during simulation can beloaded into the chip without replacing the code that is already runningin the IC. In addition, new views can be created to monitor foradditional events that were not considered during the simulation of thedesign.

For instance, an IC that includes reconfigurable circuits and aconfiguration and monitoring network can be used in a networkingequipment. After the equipment is deployed in the field, the equipmentcan be targeted for a Denial of Service (“DoS”) attack by exploiting acertain vulnerability. Malicious attackers routinely try and find newand ingenious ways to exploit vulnerabilities; as such, differentattacks can occur over time, new vulnerabilities are exploited. A viewcan be created after the equipment to check for conditions that confirmthe equipment is under DoS attack and, when those conditions aresatisfied, to take proper actions to defend the attack. The view is thentranslated into a bitstream to create an overlay. The overlay is thenloaded into the device in the field to monitor for the denial of serviceattacks without replacing the code on the chip or halting the operationsof the chip.

FIGS. 33 and 34 illustrate processes to use the same assertions duringsimulation and runtime in the some embodiments of the invention withoutpermanently including those assertions in the code that is deployed inthe field, recompiling, or replacing the deployed code. FIG. 33conceptually illustrates a process 3300 to verify a design specificationduring simulation. As shown, the process compiles (at 3305) a designspecification (e.g., written in RTL). The process also compiles (at3310) a verification suite that includes test benches and assertionsthat identify properties that the design must hold.

The process then executes (at 3315) the design specification and usesthe verification suite to check the assertions. The process then ends.

FIG. 34 conceptually illustrates a process 3400 for using the sameassertions that were used during simulation in the field withoutrecompiling or replacing the code loaded in an IC in some embodiments ofthe invention. As shown, the process loads configuration data for thereconfigurable circuits of the IC and operates the IC in the field toperform the operations of the user design. In some embodiments, thedesign specification is compiled to create configuration data for thereconfigurable circuits. The configuration data is then stored, e.g., inflash memory. The configuration data is then loaded from the SRAM intothe IC at the power up to operate the IC in the field.

Process 3400 then compiles (at 3410) a subset of the verification suiteinto incremental configuration data for the IC. For instance, theprocess recompiles a portion of the verification suite that was saved byprocess 3300.

The process then loads (at 3415) the configuration data as anincremental overlay to check for the same assertions that were checkedduring simulation while the IC is performing the user design. Loadingthe incremental overlay does not require changing, recompiling, orreplacing the user design code that is running on the IC nor it requiresthe operations of the IC to be halted for the configuration data to beloaded in the IC. The process then ends.

V. Electronic System

Many of the above-described features and applications are implemented assoftware processes that are specified as a set of instructions recordedon a computer readable storage medium (also referred to as computerreadable medium, machine readable medium, machine readable storage).When these instructions are executed by one or more computational orprocessing unit(s) (e.g., one or more processors, cores of processors,or other processing units), they cause the processing unit(s) to performthe actions indicated in the instructions. Examples of computer readablemedia include, but are not limited to, CD-ROMs, flash drives, randomaccess memory (RAM) chips, hard drives, erasable programmable read onlymemories (EPROMs), 10 electrically erasable programmable read-onlymemories (EEPROMs), etc. The computer readable media does not includecarrier waves and electronic signals passing wirelessly or over wiredconnections.

In this specification, the term “software” is meant to include firmwareresiding in read-only memory or applications stored in magnetic storage,which can be read into memory for processing by a processor. Also, insome embodiments, multiple software inventions can be implemented assub-parts of a larger program while remaining distinct softwareinventions. In some embodiments, multiple software inventions can alsobe implemented as separate programs.

Finally, any combination of separate programs that together implement asoftware invention described here is within the scope of the invention.In some embodiments, the software programs, when installed to operate onone or more electronic systems, define one or more specific machineimplementations that execute and perform the operations of the softwareprograms.

FIG. 36 conceptually illustrates an electronic system 3600 with whichsome embodiments of the invention are implemented. The electronic system3600 may be a computer (e.g., a desktop computer, personal computer,tablet computer, etc.), phone, PDA, or any other sort of electronic orcomputing device. Such an electronic system includes various types ofcomputer readable media and interfaces for various other types ofcomputer readable media. Electronic system 3600 includes a bus 3605,processing unit(s) 3610, a system memory 3620, a network 3625, aread-only memory 3630, a permanent storage device 3635, input devices3640, and output devices 3645.

The bus 3605 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of theelectronic system 3600. For instance, the bus 3605 communicativelyconnects the processing unit(s) 3610 with the read-only memory 3630, thesystem memory 3620, and the permanent storage device 3635.

From these various memory units, the processing unit(s) 3610 retrievesinstructions to execute and data to process in order to execute theprocesses of the invention. The processing unit(s) may be a singleprocessor or a multi-core processor in different embodiments. Theread-only-memory (ROM) 3630 stores static data and instructions that areneeded by the processing unit(s) 3610 and other modules of theelectronic system. The permanent storage device 3635, on the other hand,is a read-and-write memory device. This device is a non-volatile memoryunit that stores instructions and data even when the electronic system3600 is off. Some embodiments of the invention use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) asthe permanent storage device 3635.

Other embodiments use a removable storage device (such as a floppy disk,flash memory device, etc., and its corresponding disk drive) as thepermanent storage device. Like the permanent storage device 3635, thesystem memory 3620 is a read-and-write memory device. However, unlikestorage device 3635, the system memory 3620 is a volatile read-and-writememory, such a random access memory. The system memory 3620 stores someof the instructions and data that the processor needs at runtime. Insome embodiments, the invention's processes are stored in the systemmemory 3620, the permanent storage device 3635, and/or the read-onlymemory 3630. For example, the various memory units include instructionsfor processing multimedia clips in accordance with some embodiments.From these various memory units, the processing unit(s) 3610 retrievesinstructions to execute and data to process in order to execute theprocesses of some embodiments.

The bus 3605 also connects to the input devices 3640 and output devices3645. The input devices 3640 enable the user to communicate informationand select commands to the electronic system. The input devices 3640include alphanumeric keyboards and pointing devices (also called “cursorcontrol devices”), cameras (e.g., webcams), microphones or similardevices for receiving voice commands, etc. The output devices 3645display images generated by the electronic system or otherwise outputdata. The output devices 3645 include printers and display devices, suchas cathode ray tubes (CRT) or liquid crystal displays (LCD), as well asspeakers or similar audio output devices. Some embodiments includedevices such as a touchscreen that function as both input and outputdevices.

Finally, as shown in FIG. 36, bus 3605 also couples electronic system3600 to a network 3625 through a network adapter (not shown). In thismanner, the computer can be a part of a network of computers (such as alocal area network (“LAN”), a wide area network (“WAN”), or an Intranet.or a network of networks, such as the Internet. Any or all components ofelectronic system 3600 may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). Some examples of such computer-readablemedia include RAM, ROM, read-only compact discs (CD-ROM), recordablecompact discs (CD-R), rewritable compact discs (CD-RW), read-onlydigital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a varietyof recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.),flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.),magnetic and/or solid state hard drives, read-only and recordableBlu-Ray® discs, ultra density optical discs, any other optical ormagnetic media, and floppy disks. The computer-readable media may storea computer program that is executable by at least one processing unitand includes sets of instructions for performing various operations.Examples of computer programs or computer code include machine code,such as is produced by a compiler, and files including higher-level codethat are executed by a computer, an electronic component, or amicroprocessor using an interpreter.

While the above discussion primarily refers to microprocessor ormulti-core processors that execute software, some embodiments areperformed by one or more integrated circuits, such as applicationspecific integrated circuits (ASICs) or field programmable gate arrays(FPGAs). In some embodiments, such integrated circuits executeinstructions that are stored on the circuit itself. In addition, someembodiments execute software stored in programmable logic devices(PLDs), ROM, or RAM devices.

As used in this specification and any claims of this application, theterms “computer”, “server”. “processor”, and “memory” all refer toelectronic or other technological devices. These terms exclude people orgroups of people. For the purposes of the specification, the termsdisplay or displaying means displaying on an electronic device. As usedin this specification and any claims of this application, the terms“computer readable medium,” “computer readable media,” and “machinereadable medium” are entirely restricted to tangible, physical objectsthat store information in a form that is readable by a computer. Theseterms exclude any wireless signals, wired download signals, and anyother ephemeral signals.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. In addition, a number of the figures(including FIGS. 12, 18, 23-28, 33, and 34) conceptually illustrateprocesses. The specific operations of these processes may not beperformed in the exact order shown and described. The specificoperations may not be performed in one continuous series of operations,and different specific operations may be performed in differentembodiments. Furthermore, the process could be implemented using severalsub-processes, or as part of a larger macro process. Thus, one ofordinary skill in the art would understand that the invention is not tobe limited by the foregoing illustrative details, but rather is to bedefined by the appended claims.

What is claimed is:
 1. A tangible, non-transitory, machine-readablemedium, comprising machine-readable instructions to: establish a networkconnection to a remote programmable system that comprises a programmablelogic device over a network; send programming data over the networkconnection, wherein the programming data comprises at least one debuginstruction comprising a trigger condition, and wherein the at least onedebug instruction is configured to cause the programmable logic deviceto generate debug information upon meeting the trigger condition; andreceive the debug information from the remote programmable logic deviceover the network connection.
 2. The machine-readable medium of claim 1,wherein the remote programmable logic device comprises a controllercoupled to the programmable logic device.
 3. The machine-readable mediumof claim 2, comprising a memory comprising instructions to cause thecontroller to: receive the configuration data comprising the at leastone debug instruction; and provide the configuration data to theprogrammable logic device using a configuration protocol.
 4. Themachine-readable medium of claim 3, wherein the network connectioncomprises an Ethernet protocol, and wherein the configuration protocolcomprises a JTAG protocol.
 5. The machine-readable medium of claim 1,wherein the electrical device is disposed in an end-user environment. 6.The machine-readable medium of claim 1, wherein the electrical device isdisposed in a data center.
 7. The machine-readable medium of claim 1,wherein the trigger condition comprises a data capture condition, andwherein generating the debug information comprises collecting andstoring data in a buffer of the remote programmable logic device basedon an evaluation of the data capture conditions.
 8. A method,comprising: establishing a connection between a configuration device anda remote field programmable gate array device, wherein the connectionutilizes an internet-protocol network, wherein the configuration deviceand the remote field programmable gate array are not directly coupled bya configuration cable; sending instructions via the network connectionover the internet-protocol network, wherein the instructions comprise atleast one trigger condition, and wherein upon meeting the triggercondition, the instructions are configured to cause the remote fieldprogrammable gate array device to generate capture data that comprises astored data sample of a state of the remote field programmable gatearray; and receiving the capture data from the remote field programmablegate array device via the network connection.
 9. The method of claim 8,wherein sending the instructions comprise sending instructions in theinternet-protocol network to a processor coupled to a controller that iscoupled to the field programmable gate array via a configuration cable.10. The method of claim 9, wherein the configuration cable comprises aJTAG cable.
 11. The method of claim 8, wherein the internet-protocolcomprises an Ethernet protocol.
 12. The method of claim 8, wherein theinstructions comprise monitoring a signal of the field programmable gatearray using a monitor network of the remote field programmable gatearray.
 13. The method of claim 8, wherein the at least one triggercondition comprises a state machine.
 14. The method of claim 8, whereinthe field-programmable gate array comprises monitoring circuitry, andwherein the instructions comprise instructions to configure themonitoring circuitry.
 15. The method of claim 12, comprising sending asecond set of instructions via the network connection, wherein thesecond set of instructions is based in part by the received captureddata.
 16. An electronic device comprising: a programmable logic device;and a controller coupled to the programmable logic device via aconfiguration link, wherein the controller is configured to: couple to aremote server over a network connection; receive instructions to programthe programmable logic device from the remote server via the networkconnection, wherein the instructions to program the programmable logicdevice comprise at least one debug instruction and at least one triggercondition, and wherein the at least one debug instruction causes theremote programmable logic device to generate debug information u onmeeting the at least one trigger condition; and transmit the debuginformation to the remote server via the network connection.
 17. Theelectronic device of claim 16, wherein the controller comprises aprocessor.
 18. The electronic device of claim 16, wherein the electricdevice is a data center device.
 19. The electronic device of claim 16,wherein the electronic device comprises a system on chip that comprisesthe programmable logic device, and wherein the programmable logic deviceis not directly accessible by the remote server.
 20. The electronicdevice of claim 16, wherein the configuration link comprises a JTAGlink, and wherein the network connection comprises an internet-basedprotocol.
 21. A system comprising: an electronic device comprising: aprogrammable logic device configured to perform at least one customoperation; and a controller coupled to the programmable logic device viaa configuration link; and a remote host that comprises a processor and amemory device comprising instructions that cause the processor to:establish a network connection to the controller of the electronicdevice over a network connection; and send programming data over thenetwork connection to the controller, wherein the programming datacomprises at least one instruction and at least one trigger conditionassociated with the at least one custom operation, and wherein the atleast one instruction is configured to cause the programmable logicdevice to generate debug information for the at least one customoperation upon identifying a violation of the at least one triggercondition.